Capture library subtrees with downloaded audio

W on a downloadable library node mirrors the subtree into /captures
(a fourth fsdy instance) like a bookmark, but downloads every track's
audio next to its toml; the toml points at the sibling by relative
name, so captures play with no provider round trip. Nodes opt in via
the new is_downloadable flags — Tidal blesses queueable and
track-listing nodes. The bookmark walk is now the shared capture walk
parameterized by a per-track sink.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Test User 2026-07-21 13:45:04 +02:00
parent 655e8054b8
commit 7f0e900c56
23 changed files with 1702 additions and 142 deletions

1
Cargo.lock generated
View File

@ -748,6 +748,7 @@ dependencies = [
"fsdy", "fsdy",
"futures", "futures",
"rand 0.10.2", "rand 0.10.2",
"reqwest",
"serde", "serde",
"tempfile", "tempfile",
"thiserror 2.0.19", "thiserror 2.0.19",

213
architecture/captures.md Normal file
View File

@ -0,0 +1,213 @@
# Captures (downloaded subtrees)
## Context and problem statement
Bookmarks (`w`) mirror a library subtree as **link** files — replaying a
bookmark still needs the original provider. The user wants `W` on a library
node to do the same capture into a separate local provider called
**captures**, except each track's audio is **downloaded** next to its
`.cbd-track.toml`, and the toml points at that file. Playback of a capture
then needs no provider round trip at all — it is a fully local copy.
A library node decides whether it allows `W`; Tidal implements it.
## Assumptions (confirmed against the code)
- `PlayableSpec.file` already supports **relative** paths, resolved against
the track file's directory at `get_urls_for_track` time (fs-provider D3).
A toml next to its audio file can say `file = "0001 Song.flac"` and the
whole capture folder stays relocatable (tmp-and-swap, rename, backup).
- `fsdy::list_dir` only surfaces directories and `*.cbd-track.toml` files;
downloaded audio siblings are invisible to the library listing.
- The bookmark walk (`bookmark_store::write_capture`) is an iterative
pre-order worklist whose only per-track action is "serialize and write
one file" — exactly the seam where a download variant plugs in.
- `reqwest` (rustls, `stream`) is already a workspace dependency; tidal
stream URLs come from `get_urls_for_track` on the orchestrator, so the
download needs no new provider methods.
- Uppercase bindings (`K`, `J`) already exist in the TUI bindings table;
`W` in `Scope::Library` is free.
- `LibraryNode`/`LibraryNodeChild` already model per-node capabilities
(`is_queable`, `is_creatable`, …) — the "does this node allow `W`"
decision extends that pattern.
## Decisions
### D1 — Fourth `fsdy` instance at `/captures`
`<config>/crabidy/captures/` is mounted read-only as `/captures` with an
editable top level (no reserved names), exactly like `/bookmarks`. Init is
non-fatal: an unopenable store disables `W` and the mount, never the
server. Loading a capture is browsing `/captures` and queueing a folder —
zero new replay mechanisms.
### D2 — One shared walk, two track sinks
Options considered:
- *(a)* Copy `bookmark_store.rs` and swap the per-track write.
- *(b)* Extract the walk into a shared `capture` module parameterized by a
**track sink**; bookmarks and captures become thin stores over it.
**Decision: (b)** — the walk (worklist, caps, temp-and-swap, all-or-nothing
cleanup, `BadSource` mapping) is behavior we already tested once and must
not fork. `crabidy-server/src/capture.rs` owns `CaptureError`, the caps,
and `write_tree(client, source, tmp, caps, sink)`; the sink is an enum
(no async-trait indirection):
- `Sink::Link` — today's bookmark behavior, byte-identical
(`TrackFile::from_track`, link playable).
- `Sink::Download(Downloader)` — captures (D3).
`BookmarkStore` keeps its API; `CaptureStore` (in `capture_store.rs`) is
its sibling over the captures directory.
### D3 — Download sink: audio next to the toml, toml points at it
Per track, in listing order:
1. `get_urls_for_track` through the orchestrator (any provider that yields
URLs works; Tidal is the target). First URL wins.
2. HTTP GET via one shared `reqwest` client — connect timeout, one total
per-track deadline covering the whole body, **no retries** (a capture
is re-runnable and overwrite = refresh; a retry policy can come later).
The body is streamed to `NNNN <title>.<ext>` (shared `ordered_name`
sanitizer, same 4-digit prefix as the toml so the pair sorts together).
3. The extension comes from the response `Content-Type`
(`audio/flac` → `flac`, `audio/mp4`/`audio/m4a` → `m4a`,
`audio/mpeg``mp3`, `audio/ogg``ogg`, `audio/wav``wav`),
falling back to the URL path's extension, then `bin` (the player probes
by content; the extension is a hint).
4. The toml is written **after** the download succeeds, with
`playable.file = "<audio file name>"` (relative, new
`TrackFile::from_track_with_file`), keeping metadata identical to a
bookmark entry.
Caps: `MAX_CAPTURE_DIRS` stays 1 000; downloads get their own
`MAX_DOWNLOAD_TRACKS = 500` and a total byte budget
`MAX_DOWNLOAD_BYTES = 4 GiB` counted while streaming — a runaway artist
capture must not fill the disk. Downloads run sequentially (gentle on the
provider, trivially bounded memory); the whole capture already runs on a
spawned task, so the orchestrator keeps serving.
All-or-nothing is kept: any failed download aborts the capture and removes
the temp folder. A capture that lists 30 tracks *has* 30 playable files.
### D4 — Nodes opt in via `is_downloadable`
New proto fields `LibraryNode.is_downloadable = 8` and
`LibraryNodeChild.is_downloadable = 7` (additive). Tidal sets the flag
centrally at the end of `get_lib_node`: a node is downloadable when it is
**queueable or lists tracks** (the "or lists tracks" covers search-term
result nodes, which are not queueable as a whole but whose track results
are downloadable); children mirror `is_queable`. Everything else (fs,
queues, bookmarks, captures, orchestrator roots) leaves the default
`false`; capturing a capture is pointless and links must not masquerade
as downloads.
Tracks carry no flag: a listed track inherits its containing node's
`is_downloadable` (TUI) — a Tidal album's tracks are downloadable because
the album is. The server enforces at the capture **root**: a directory
source must report `is_downloadable`, a track source's parent node must
(`CaptureError::Unsupported` otherwise). Nested nodes inside the walk are
not re-checked — the pressed node's decision governs its subtree.
### D5 — Wire: the existing rpc gains a `download` flag
`CaptureLibraryNodeRequest` gets `bool download = 3` (additive; old
clients keep bookmarking). `ProviderCommand::CaptureLibraryNode` carries
it and the handler picks the store. Error mapping extends the bookmark
contract: `InvalidName`/`BadSource` → `invalid_argument`,
`TooLarge`/`Disabled`/`Unsupported` → `failed_precondition`, download and
disk failures → `internal`.
### D6 — TUI: `W` on the library pane
`Action::LibraryDownloadNode` bound to `W` in `Scope::Library` ("Download
selection as capture"). Gate: the bare selection must be queueable **and**
downloadable (`selected_downloadable()`; child flag for nodes, the current
node's flag for tracks; marks ignored like `w`). The existing input
overlay opens with `InputPurpose::Capture { path, download: true }`
(label `capture`), prefilled with the selection title. Submit sends
`MessageFromUi::CaptureNode { path, name, download }` → the rpc. Failures
are logged, never fatal to the poll loop.
### D7 — Out of scope (explicitly)
- Retry/resume of failed or partial downloads (re-run the capture).
- Quality/codec selection, transcoding, tagging the audio files.
- Progress display in the TUI while a capture downloads.
- Deduplicating audio across captures, or refreshing links in existing
bookmarks into downloads.
- DRM circumvention: the download uses exactly the stream URLs the
provider already serves for playback.
## Structure
```d2
direction: right
server: crabidy-server {
orch: ProviderOrchestrator
cap: "capture.rs\nshared walk + caps + swap" {
link: "Sink::Link"
dl: "Sink::Download\n(reqwest, timeouts, byte budget)"
}
bs: BookmarkStore
cs: CaptureStore
}
tidal: "tidaldy\n(is_downloadable = is_queable)"
disk: "config/crabidy" {
shape: cylinder
b: "bookmarks/<name>/ (link tomls)"
c: "captures/<name>/ (audio + file tomls)"
}
cfs: "fsdy /captures\n(editable top level)"
server.orch -> server.bs: "CaptureLibraryNode\ndownload=false"
server.orch -> server.cs: "CaptureLibraryNode\ndownload=true"
server.bs -> server.cap.link
server.cs -> server.cap.dl
server.cap.dl -> tidal: "get_urls_for_track\n+ HTTP GET stream"
server.bs -> disk.b
server.cs -> disk.c
cfs -> disk.c: "list + parse (read only)"
server.orch -> cfs: "/captures/..."
```
## Key flow: W on a Tidal album
```d2
shape: sequence_diagram
tui: TUI
rpc: gRPC
orch: Orchestrator
cs: CaptureStore
tidal: Tidal
tui -> rpc: "CaptureLibraryNode(path, name, download=true)"
rpc -> orch: "ProviderCommand (spawned)"
orch -> cs: "capture(name)"
cs -> orch: "get_lib_node: root allows download?"
cs -> tidal: "per track: get_urls_for_track"
cs -> tidal: "HTTP GET (deadline, byte budget)"
cs -> cs: "audio + toml pair\n(toml after audio, file = relative)"
cs -> rpc: "tmp-and-swap captures/<name>/"
rpc -> tui: OK
```
## Risks and open questions
- **Disk usage**: 500 tracks of FLAC can be tens of GiB; the byte budget
caps one capture, not the folder's total. Accepted — the user manages
`captures/` like any local music folder (and can delete via `d`).
- **Stream URL churn**: Tidal URLs are short-lived; the download happens
immediately after fetching each URL, so expiry only matters for very
slow transfers, which the per-track deadline already bounds.
- **Licensing**: captures are personal-use copies of streams the account
can already play; nothing here bypasses provider protection.
- Open (future): a progress event stream for long captures; retry with
classification + jitter; per-provider download quality knobs.

View File

@ -72,6 +72,10 @@ pub enum Action {
/// capture the selected queueable subtree as a bookmark under /// capture the selected queueable subtree as a bookmark under
/// `/bookmarks`. No-op unless the bare selection `is_queable`. /// `/bookmarks`. No-op unless the bare selection `is_queable`.
LibraryCaptureNode, LibraryCaptureNode,
/// Like [`Self::LibraryCaptureNode`], but the capture **downloads**
/// every track's audio into `/captures`. No-op unless the bare
/// selection is queueable *and* downloadable (e.g. Tidal subtrees).
LibraryDownloadNode,
// Queue pane // Queue pane
QueueInsertHere, QueueInsertHere,
QueueFirst, QueueFirst,
@ -266,6 +270,13 @@ pub const BINDINGS: &[Binding] = &[
action: Action::LibraryCaptureNode, action: Action::LibraryCaptureNode,
description: "Save selection as bookmark", description: "Save selection as bookmark",
}, },
Binding {
scope: Scope::Library,
mods: KeyModifiers::SHIFT,
code: KeyCode::Char('W'),
action: Action::LibraryDownloadNode,
description: "Download selection as capture",
},
Binding { Binding {
scope: Scope::Library, scope: Scope::Library,
mods: KeyModifiers::NONE, mods: KeyModifiers::NONE,
@ -563,6 +574,23 @@ mod tests {
), ),
Some(Action::LibraryCaptureNode) Some(Action::LibraryCaptureNode)
); );
// Shift-w is the download capture, library pane only.
assert_eq!(
lookup(
UiFocus::Library,
false,
key(KeyCode::Char('W'), KeyModifiers::SHIFT)
),
Some(Action::LibraryDownloadNode)
);
assert_eq!(
lookup(
UiFocus::Queue,
false,
key(KeyCode::Char('W'), KeyModifiers::SHIFT)
),
None
);
} }
#[test] #[test]

View File

@ -74,6 +74,13 @@ impl Library {
item.is_queable item.is_queable
.then(|| (item.path.clone(), item.title.clone())) .then(|| (item.path.clone(), item.title.clone()))
} }
/// The bare selection's path and title when it can be captured with a
/// download (`W`): queueable *and* downloadable. Marks are ignored,
/// like [`Self::selected_queueable`].
pub fn selected_downloadable(&self) -> Option<(String, String)> {
let item = self.list.get(self.list_state.selected()?)?;
(item.is_queable && item.is_downloadable).then(|| (item.path.clone(), item.title.clone()))
}
pub fn get_selected(&self) -> Option<Vec<String>> { pub fn get_selected(&self) -> Option<Vec<String>> {
if self.list.iter().any(|i| i.marked) { if self.list.iter().any(|i| i.marked) {
return Some( return Some(
@ -188,6 +195,9 @@ impl Library {
is_creatable: false, is_creatable: false,
is_editable: false, is_editable: false,
is_deletable: false, is_deletable: false,
// Tracks carry no wire flag: they inherit their node's
// blessing (architecture/captures.md D4).
is_downloadable: node.is_downloadable,
}) })
.chain(node.children.iter().map(|c| UiItem { .chain(node.children.iter().map(|c| UiItem {
path: c.path.clone(), path: c.path.clone(),
@ -198,6 +208,7 @@ impl Library {
is_creatable: c.is_creatable, is_creatable: c.is_creatable,
is_editable: c.is_editable, is_editable: c.is_editable,
is_deletable: c.is_deletable, is_deletable: c.is_deletable,
is_downloadable: c.is_downloadable,
})) }))
.collect(); .collect();

View File

@ -48,6 +48,9 @@ struct UiItem {
is_editable: bool, is_editable: bool,
/// This item may be deleted (`d`) — part of the `[ed]` marker. /// This item may be deleted (`d`) — part of the `[ed]` marker.
is_deletable: bool, is_deletable: bool,
/// This item allows download captures (`W`). Tracks inherit their
/// containing node's flag; child nodes carry their own.
is_downloadable: bool,
} }
pub const COLOR_PRIMARY: Color = Color::Rgb(129, 161, 193); pub const COLOR_PRIMARY: Color = Color::Rgb(129, 161, 193);
@ -90,10 +93,13 @@ pub enum MessageFromUi {
/// as `/queues/<name>` in the library on the next visit). /// as `/queues/<name>` in the library on the next visit).
SaveQueue(String), SaveQueue(String),
/// Capture the queueable subtree at `path` as the bookmark `name` /// Capture the queueable subtree at `path` as the bookmark `name`
/// (structure-preserving snapshot under `/bookmarks/<name>`). /// (structure-preserving snapshot under `/bookmarks/<name>`), or —
/// with `download` — as the capture `name` under `/captures/<name>`
/// with every track's audio downloaded next to its toml.
CaptureNode { CaptureNode {
path: String, path: String,
name: String, name: String,
download: bool,
}, },
AppendTracks(Vec<String>), AppendTracks(Vec<String>),
QueueTracks(Vec<String>), QueueTracks(Vec<String>),
@ -135,8 +141,10 @@ pub enum InputPurpose {
/// `w`: save the current queue under the entered name. /// `w`: save the current queue under the entered name.
SaveQueue, SaveQueue,
/// `w` in the library: capture the queueable subtree at `path` as a /// `w` in the library: capture the queueable subtree at `path` as a
/// bookmark. The buffer starts prefilled with the selection's title. /// bookmark — or, for `W` (`download`), as a download capture (the
Capture { path: String }, /// selection must also be downloadable). The buffer starts prefilled
/// with the selection's title.
Capture { path: String, download: bool },
} }
/// State of the one-line text input overlay (node creation and rename). /// State of the one-line text input overlay (node creation and rename).
@ -213,10 +221,11 @@ impl App {
InputPurpose::SaveQueue => { InputPurpose::SaveQueue => {
let _ = self.tx.send(MessageFromUi::SaveQueue(title)); let _ = self.tx.send(MessageFromUi::SaveQueue(title));
} }
InputPurpose::Capture { path } => { InputPurpose::Capture { path, download } => {
let _ = self.tx.send(MessageFromUi::CaptureNode { let _ = self.tx.send(MessageFromUi::CaptureNode {
path: path.clone(), path: path.clone(),
name: title, name: title,
download: *download,
}); });
} }
} }
@ -312,7 +321,23 @@ impl App {
// non-queueable selections like the other gated openers. // non-queueable selections like the other gated openers.
if let Some((path, title)) = self.library.selected_queueable() { if let Some((path, title)) = self.library.selected_queueable() {
self.input = Some(InputState { self.input = Some(InputState {
purpose: InputPurpose::Capture { path }, purpose: InputPurpose::Capture {
path,
download: false,
},
buffer: title,
});
}
}
Action::LibraryDownloadNode => {
// `W`: like `w`, but the selection must also allow download
// captures (architecture/captures.md D6).
if let Some((path, title)) = self.library.selected_downloadable() {
self.input = Some(InputState {
purpose: InputPurpose::Capture {
path,
download: true,
},
buffer: title, buffer: title,
}); });
} }
@ -389,7 +414,10 @@ impl App {
InputPurpose::Create { .. } => "new node", InputPurpose::Create { .. } => "new node",
InputPurpose::Rename { .. } => "rename", InputPurpose::Rename { .. } => "rename",
InputPurpose::SaveQueue => "save queue", InputPurpose::SaveQueue => "save queue",
InputPurpose::Capture { .. } => "bookmark", InputPurpose::Capture {
download: false, ..
} => "bookmark",
InputPurpose::Capture { download: true, .. } => "capture",
}; };
let line = Rect::new(area.x + 1, area.y + area.height - 2, area.width - 2, 1); let line = Rect::new(area.x + 1, area.y + area.height - 2, area.width - 2, 1);
f.render_widget(Clear, line); f.render_widget(Clear, line);
@ -482,6 +510,7 @@ mod tests {
tracks: Vec::new(), tracks: Vec::new(),
is_queable: false, is_queable: false,
is_creatable: true, is_creatable: true,
is_downloadable: false,
} }
} }
@ -793,9 +822,13 @@ mod tests {
app.library.update(queueable_listing()); app.library.update(queueable_listing());
let _ = app.dispatch(Action::LibraryCaptureNode); let _ = app.dispatch(Action::LibraryCaptureNode);
let input = app.input.as_ref().expect("capture overlay open"); let input = app.input.as_ref().expect("capture overlay open");
assert!( assert!(matches!(
matches!(&input.purpose, InputPurpose::Capture { path } if path == "/tidal/artists/1") &input.purpose,
); InputPurpose::Capture {
path,
download: false
} if path == "/tidal/artists/1"
));
assert_eq!(input.buffer, "artist", "prefilled with the title"); assert_eq!(input.buffer, "artist", "prefilled with the title");
} }
@ -809,9 +842,14 @@ mod tests {
app.handle_input_key(key(crossterm::event::KeyCode::Enter)); app.handle_input_key(key(crossterm::event::KeyCode::Enter));
assert!(app.input.is_none()); assert!(app.input.is_none());
match rx.try_recv() { match rx.try_recv() {
Ok(MessageFromUi::CaptureNode { path, name }) => { Ok(MessageFromUi::CaptureNode {
path,
name,
download,
}) => {
assert_eq!(path, "/tidal/artists/1"); assert_eq!(path, "/tidal/artists/1");
assert_eq!(name, "artist favs"); assert_eq!(name, "artist favs");
assert!(!download, "w captures a bookmark, not a download");
} }
other => panic!("expected CaptureNode, got {:?}", other.is_ok()), other => panic!("expected CaptureNode, got {:?}", other.is_ok()),
} }
@ -822,6 +860,95 @@ mod tests {
assert!(rx.try_recv().is_err(), "Esc must not capture"); assert!(rx.try_recv().is_err(), "Esc must not capture");
} }
/// Like [`queueable_listing`], but the child also allows download
/// captures (a Tidal subtree).
fn downloadable_listing() -> LibraryNode {
use crabidy_core::proto::crabidy::LibraryNodeChild;
LibraryNode {
children: vec![LibraryNodeChild {
is_downloadable: true,
..LibraryNodeChild::new("/tidal/artists/1".to_string(), "artist".to_string(), true)
}],
..creatable_node("/tidal/artists")
}
}
#[test]
fn download_capture_requires_the_downloadable_flag() {
let (mut app, _rx) = app();
// Queueable but not downloadable (e.g. an /fs folder): `W` must
// stay closed even though `w` would open.
app.library.update(queueable_listing());
let _ = app.dispatch(Action::LibraryDownloadNode);
assert!(app.input.is_none());
app.library.update(downloadable_listing());
let _ = app.dispatch(Action::LibraryDownloadNode);
let input = app.input.as_ref().expect("download overlay open");
assert!(matches!(
&input.purpose,
InputPurpose::Capture {
path,
download: true
} if path == "/tidal/artists/1"
));
assert_eq!(input.buffer, "artist", "prefilled with the title");
}
#[test]
fn tracks_inherit_their_nodes_download_blessing() {
use crabidy_core::proto::crabidy::Track;
let track = Track {
path: "/tidal/artists/1/2/3".to_string(),
artist: "a".to_string(),
title: "one".to_string(),
duration: None,
album: None,
};
let listing = |downloadable| LibraryNode {
tracks: vec![track.clone()],
is_downloadable: downloadable,
..creatable_node("/tidal/artists/1/2")
};
let (mut app, _rx) = app();
app.library.update(listing(false));
let _ = app.dispatch(Action::LibraryDownloadNode);
assert!(app.input.is_none(), "unblessed node, unblessed tracks");
app.library.update(listing(true));
let _ = app.dispatch(Action::LibraryDownloadNode);
let input = app.input.as_ref().expect("download overlay open");
assert!(matches!(
&input.purpose,
InputPurpose::Capture {
path,
download: true
} if path == "/tidal/artists/1/2/3"
));
}
#[test]
fn download_capture_submits_with_the_download_flag() {
let (mut app, rx) = app();
app.library.update(downloadable_listing());
let _ = app.dispatch(Action::LibraryDownloadNode);
type_str(&mut app, " local ");
app.handle_input_key(key(crossterm::event::KeyCode::Enter));
assert!(app.input.is_none());
match rx.try_recv() {
Ok(MessageFromUi::CaptureNode {
path,
name,
download,
}) => {
assert_eq!(path, "/tidal/artists/1");
assert_eq!(name, "artist local");
assert!(download, "W captures a download");
}
other => panic!("expected CaptureNode, got {:?}", other.is_ok()),
}
}
/// A one-track queue update, as the server would broadcast it. /// A one-track queue update, as the server would broadcast it.
fn one_track_queue() -> crabidy_core::proto::crabidy::Queue { fn one_track_queue() -> crabidy_core::proto::crabidy::Queue {
use crabidy_core::proto::crabidy::{Queue as ProtoQueue, Track}; use crabidy_core::proto::crabidy::{Queue as ProtoQueue, Track};

View File

@ -81,6 +81,7 @@ impl Queue {
is_creatable: false, is_creatable: false,
is_editable: false, is_editable: false,
is_deletable: false, is_deletable: false,
is_downloadable: false,
}) })
.collect(); .collect();

View File

@ -203,11 +203,14 @@ async fn poll(
error!(name, "failed to save queue: {err}"); error!(name, "failed to save queue: {err}");
} }
} }
MessageFromUi::CaptureNode { path, name } => { MessageFromUi::CaptureNode { path, name, download } => {
// A rejected capture (bad name, over-cap subtree) must // A rejected capture (bad name, over-cap subtree, failed
// not tear down the poll loop either. // download) must not tear down the poll loop either.
if let Err(err) = rpc_client.capture_library_node(path.clone(), name.clone()).await { if let Err(err) = rpc_client
error!(path, name, "failed to capture subtree: {err}"); .capture_library_node(path.clone(), name.clone(), download)
.await
{
error!(path, name, download, "failed to capture subtree: {err}");
} }
} }
} }

View File

@ -225,8 +225,13 @@ impl RpcClient {
&mut self, &mut self,
path: String, path: String,
name: String, name: String,
download: bool,
) -> Result<(), Box<dyn Error>> { ) -> Result<(), Box<dyn Error>> {
let capture_request = Request::new(CaptureLibraryNodeRequest { path, name }); let capture_request = Request::new(CaptureLibraryNodeRequest {
path,
name,
download,
});
self.client.capture_library_node(capture_request).await?; self.client.capture_library_node(capture_request).await?;
Ok(()) Ok(())
} }

View File

@ -105,8 +105,12 @@ message DeleteLibraryNodeResponse {
message CaptureLibraryNodeRequest { message CaptureLibraryNodeRequest {
// Path of the queueable node (or track) to capture. // Path of the queueable node (or track) to capture.
string path = 1; string path = 1;
// Bookmark name; becomes the top-level folder under /bookmarks. // Capture name; becomes the top-level folder under /bookmarks (or
// /captures when download is set).
string name = 2; string name = 2;
// Download every track's audio into the capture instead of writing
// link files; the source node must set is_downloadable.
bool download = 3;
} }
message CaptureLibraryNodeResponse {} message CaptureLibraryNodeResponse {}
@ -207,6 +211,8 @@ message LibraryNodeChild {
bool is_editable = 5; bool is_editable = 5;
// This node may be deleted (see DeleteLibraryNode). // This node may be deleted (see DeleteLibraryNode).
bool is_deletable = 6; bool is_deletable = 6;
// This node allows download captures (CaptureLibraryNode with download).
bool is_downloadable = 7;
} }
message QueueModifiers { message QueueModifiers {
@ -268,4 +274,7 @@ message LibraryNode {
bool is_queable = 6; bool is_queable = 6;
// Children may be created under this node (see CreateLibraryNode). // Children may be created under this node (see CreateLibraryNode).
bool is_creatable = 7; bool is_creatable = 7;
// This node allows download captures; its listed tracks inherit the
// flag (CaptureLibraryNode with download).
bool is_downloadable = 8;
} }

View File

@ -209,14 +209,16 @@ impl LibraryNode {
tracks: Vec::new(), tracks: Vec::new(),
is_queable: false, is_queable: false,
is_creatable: false, is_creatable: false,
is_downloadable: false,
} }
} }
} }
impl LibraryNodeChild { impl LibraryNodeChild {
/// A regular, non-creatable, immutable child. Creatable/editable/ /// A regular, non-creatable, immutable child. Creatable/editable/
/// deletable children (e.g. the search node and its terms) set the /// deletable/downloadable children (e.g. the search node and its
/// capability flags explicitly via struct update. /// terms, or Tidal's queueable subtrees) set the capability flags
/// explicitly via struct update.
pub fn new(path: String, title: String, is_queable: bool) -> Self { pub fn new(path: String, title: String, is_queable: bool) -> Self {
Self { Self {
path, path,
@ -225,6 +227,7 @@ impl LibraryNodeChild {
is_creatable: false, is_creatable: false,
is_editable: false, is_editable: false,
is_deletable: false, is_deletable: false,
is_downloadable: false,
} }
} }
} }
@ -361,6 +364,7 @@ mod tests {
.collect(), .collect(),
is_queable, is_queable,
is_creatable: false, is_creatable: false,
is_downloadable: false,
} }
} }

View File

@ -17,6 +17,7 @@ flume.workspace = true
fsdy.workspace = true fsdy.workspace = true
futures.workspace = true futures.workspace = true
rand.workspace = true rand.workspace = true
reqwest.workspace = true
serde.workspace = true serde.workspace = true
thiserror.workspace = true thiserror.workspace = true
tidaldy.workspace = true tidaldy.workspace = true

View File

@ -17,10 +17,10 @@ pub const BOOKMARKS_PROVIDER_ROOT: &str = "/bookmarks";
/// The walk aborts beyond this many directories — a runaway provider tree /// The walk aborts beyond this many directories — a runaway provider tree
/// must not fill the disk. /// must not fill the disk.
pub const MAX_CAPTURE_DIRS: usize = 1_000; pub const MAX_CAPTURE_DIRS: usize = crate::capture::BOOKMARK_CAPS.max_dirs;
/// The walk aborts beyond this many track files. /// The walk aborts beyond this many track files.
pub const MAX_CAPTURE_TRACKS: usize = 20_000; pub const MAX_CAPTURE_TRACKS: usize = crate::capture::BOOKMARK_CAPS.max_tracks;
/// The bookmarks directory: `bookmarks/` inside the crabidy config /// The bookmarks directory: `bookmarks/` inside the crabidy config
/// directory. `None` when the platform has no config directory. /// directory. `None` when the platform has no config directory.
@ -28,26 +28,7 @@ pub fn bookmarks_dir() -> Option<PathBuf> {
dirs::config_dir().map(|d| d.join("crabidy").join("bookmarks")) dirs::config_dir().map(|d| d.join("crabidy").join("bookmarks"))
} }
/// Errors from validating or writing a capture. pub use crate::capture::CaptureError;
///
/// At the RPC boundary: `InvalidName`/`BadSource` → `invalid_argument`,
/// `TooLarge` → `failed_precondition`, the rest → `internal`. Messages
/// carry names and paths, never file contents.
#[derive(Debug, thiserror::Error)]
pub enum CaptureError {
#[error("invalid bookmark name: {0}")]
InvalidName(&'static str),
#[error("bookmarks are disabled")]
Disabled,
#[error("the source path cannot be captured: {0}")]
BadSource(String),
#[error("the subtree is too large to capture ({0})")]
TooLarge(&'static str),
#[error("cannot write bookmark: {0}")]
Io(#[from] std::io::Error),
#[error(transparent)]
TrackFile(#[from] fsdy::TrackFileError),
}
/// Writes captured subtrees. All I/O is `tokio::fs`; the whole bookmark is /// Writes captured subtrees. All I/O is `tokio::fs`; the whole bookmark is
/// built as a hidden temp sibling and swapped into place, so a crash never /// built as a hidden temp sibling and swapped into place, so a crash never
@ -113,88 +94,20 @@ impl BookmarkStore {
where where
C: ProviderClient + Sync, C: ProviderClient + Sync,
{ {
let name = fsdy::validate_folder_name(name, &[]).map_err(CaptureError::InvalidName)?; let caps = crate::capture::Caps {
let tmp = self.dir.join(format!(".tmp-{name}")); max_dirs,
let written = self max_tracks,
.write_capture(client, source_path, &tmp, max_dirs, max_tracks) ..crate::capture::BOOKMARK_CAPS
.await; };
if let Err(err) = written { crate::capture::capture_into(
// Every failure path removes the temp folder: nothing &self.dir,
// half-written survives, not even hidden. client,
let _ = tokio::fs::remove_dir_all(&tmp).await; source_path,
return Err(err); name,
} caps,
let target = self.dir.join(name); &crate::capture::Sink::Link,
if tokio::fs::try_exists(&target).await? { )
tokio::fs::remove_dir_all(&target).await?; .await
}
tokio::fs::rename(&tmp, &target).await?;
Ok(())
}
/// Builds the mirrored tree inside `tmp`. All-or-nothing: any provider
/// or write failure aborts the whole capture (the caller removes
/// `tmp`) — a bookmark that *looks* complete must *be* complete.
async fn write_capture<C>(
&self,
client: &C,
source_path: &str,
tmp: &Path,
max_dirs: usize,
max_tracks: usize,
) -> Result<(), CaptureError>
where
C: ProviderClient + Sync,
{
// A leftover temp folder from a crashed or racing capture is stale.
if tokio::fs::try_exists(tmp).await? {
tokio::fs::remove_dir_all(tmp).await?;
}
tokio::fs::create_dir_all(tmp).await?;
// A track source captures as a folder with one file.
if client.is_track_path(source_path) {
let track = client
.get_metadata_for_track(source_path)
.await
.map_err(|err| CaptureError::BadSource(format!("{source_path}: {err}")))?;
let text = fsdy::TrackFile::from_track(&track).to_toml()?;
let file = tmp.join(fsdy::track_file_name(0, &track.title));
tokio::fs::write(file, text).await?;
return Ok(());
}
let mut dirs = 1usize;
let mut tracks = 0usize;
// Iterative pre-order: a deep provider tree must not overflow the
// stack. Each entry pairs a library path with its mirror folder.
let mut worklist: Vec<(String, PathBuf)> =
vec![(source_path.to_string(), tmp.to_path_buf())];
while let Some((lib_path, dir)) = worklist.pop() {
let node = client
.get_lib_node(&lib_path)
.await
.map_err(|err| CaptureError::BadSource(format!("{lib_path}: {err}")))?;
for (index, track) in node.tracks.iter().enumerate() {
tracks += 1;
if tracks > max_tracks {
return Err(CaptureError::TooLarge("too many tracks"));
}
let text = fsdy::TrackFile::from_track(track).to_toml()?;
let file = dir.join(fsdy::track_file_name(index, &track.title));
tokio::fs::write(file, text).await?;
}
for (index, child) in node.children.iter().enumerate() {
dirs += 1;
if dirs > max_dirs {
return Err(CaptureError::TooLarge("too many directories"));
}
let child_dir = dir.join(fsdy::dir_name(index, &child.title));
tokio::fs::create_dir(&child_dir).await?;
worklist.push((child.path.clone(), child_dir));
}
}
Ok(())
} }
} }

View File

@ -0,0 +1,414 @@
//! The shared subtree-capture walk (see `architecture/captures.md` D2).
//!
//! Both bookmark captures (`w`, link files) and download captures (`W`,
//! audio files next to their tomls) mirror a library subtree into a folder:
//! an iterative pre-order walk over [`ProviderClient::get_lib_node`] with
//! order-prefixed names, size caps, a hidden tmp-and-swap write, and
//! all-or-nothing cleanup. This module owns that walk; the two stores only
//! differ in the per-track [`Sink`].
use std::path::{Path, PathBuf};
use crabidy_core::proto::crabidy::Track;
use crabidy_core::ProviderClient;
/// Connect timeout for download requests.
pub const DOWNLOAD_CONNECT_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
/// Total per-track deadline: URL fetch, request, and streaming the whole
/// body. A stalled transfer aborts the capture instead of hanging it.
pub const DOWNLOAD_TRACK_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(600);
/// Size limits for one capture. The walk aborts with
/// [`CaptureError::TooLarge`] when a limit trips — a runaway provider tree
/// or oversized stream must not fill the disk.
#[derive(Debug, Clone, Copy)]
pub struct Caps {
/// Maximum directories (the capture root counts as the first).
pub max_dirs: usize,
/// Maximum track files.
pub max_tracks: usize,
/// Maximum total downloaded bytes (ignored by [`Sink::Link`]).
pub max_bytes: u64,
}
/// Caps for bookmark (link) captures: link files are tiny, so only the
/// tree size is bounded.
pub const BOOKMARK_CAPS: Caps = Caps {
max_dirs: 1_000,
max_tracks: 20_000,
max_bytes: u64::MAX,
};
/// Caps for download captures: fewer tracks and a 4 GiB byte budget.
pub const DOWNLOAD_CAPS: Caps = Caps {
max_dirs: 1_000,
max_tracks: 500,
max_bytes: 4 * 1024 * 1024 * 1024,
};
/// Errors from validating or writing a capture.
///
/// At the RPC boundary: `InvalidName`/`BadSource` → `invalid_argument`,
/// `TooLarge`/`Disabled`/`Unsupported` → `failed_precondition`, the rest →
/// `internal`. Messages carry names, paths, and counts, never file
/// contents.
#[derive(Debug, thiserror::Error)]
pub enum CaptureError {
#[error("invalid name: {0}")]
InvalidName(&'static str),
#[error("the store is disabled")]
Disabled,
#[error("the source path cannot be captured: {0}")]
BadSource(String),
#[error("the source does not allow downloads")]
Unsupported,
#[error("the subtree is too large to capture ({0})")]
TooLarge(&'static str),
#[error("download failed: {0}")]
Download(String),
#[error("cannot write capture: {0}")]
Io(#[from] std::io::Error),
#[error(transparent)]
TrackFile(#[from] fsdy::TrackFileError),
}
/// What happens to each track the walk visits.
#[derive(Debug)]
pub enum Sink {
/// Write an order-prefixed link file ([`fsdy::TrackFile::from_track`])
/// — the bookmark behavior.
Link,
/// Download the track's audio next to an order-prefixed toml that
/// points at it — the captures behavior.
Download(Downloader),
}
/// Downloads one track's audio via the provider's stream URL.
///
/// One shared HTTP client with a connect timeout; each track is bounded by
/// [`DOWNLOAD_TRACK_TIMEOUT`] end to end and never retried (a capture is
/// re-runnable; overwrite = refresh). Bodies are streamed to disk against
/// the capture's remaining byte budget.
#[derive(Debug)]
pub struct Downloader {
http: reqwest::Client,
}
impl Downloader {
/// Builds the shared HTTP client. Fails only when the TLS backend
/// cannot initialize.
pub fn new() -> Result<Self, reqwest::Error> {
let http = reqwest::Client::builder()
.connect_timeout(DOWNLOAD_CONNECT_TIMEOUT)
.build()?;
Ok(Self { http })
}
/// Downloads one track: audio file first, then the toml pointing at
/// it — a toml never exists without its audio. The whole operation is
/// bounded by [`DOWNLOAD_TRACK_TIMEOUT`].
///
/// Error messages carry the track's library path, never the stream
/// URL (it may embed a token) — reqwest errors are stripped with
/// [`reqwest::Error::without_url`].
async fn download_track<C>(
&self,
client: &C,
track: &Track,
dir: &Path,
index: usize,
bytes_left: &mut u64,
) -> Result<(), CaptureError>
where
C: ProviderClient + Sync,
{
let fetched = tokio::time::timeout(
DOWNLOAD_TRACK_TIMEOUT,
self.fetch_track(client, track, dir, index, bytes_left),
)
.await;
match fetched {
Ok(result) => result,
Err(_) => Err(CaptureError::Download(format!(
"{}: timed out after {}s",
track.path,
DOWNLOAD_TRACK_TIMEOUT.as_secs()
))),
}
}
/// The unbounded body of [`Self::download_track`]: resolve the stream
/// URL, stream the response to disk against the byte budget, then
/// write the toml.
async fn fetch_track<C>(
&self,
client: &C,
track: &Track,
dir: &Path,
index: usize,
bytes_left: &mut u64,
) -> Result<(), CaptureError>
where
C: ProviderClient + Sync,
{
let track_path = track.path.as_str();
let urls = client
.get_urls_for_track(track_path)
.await
.map_err(|err| CaptureError::BadSource(format!("{track_path}: {err}")))?;
let url = urls
.first()
.ok_or_else(|| CaptureError::BadSource(format!("{track_path}: no stream url")))?;
let download_err = |err: reqwest::Error| {
CaptureError::Download(format!("{track_path}: {}", err.without_url()))
};
let mut response = self
.http
.get(url)
.send()
.await
.map_err(download_err)?
.error_for_status()
.map_err(download_err)?;
let content_type = response
.headers()
.get(reqwest::header::CONTENT_TYPE)
.and_then(|value| value.to_str().ok())
.map(str::to_string);
let ext = extension_for(content_type.as_deref(), url);
let audio_name = audio_file_name(index, &track.title, &ext);
let mut audio = tokio::fs::File::create(dir.join(&audio_name)).await?;
while let Some(chunk) = response.chunk().await.map_err(download_err)? {
let len = chunk.len() as u64;
if len > *bytes_left {
return Err(CaptureError::TooLarge("download budget exhausted"));
}
*bytes_left -= len;
tokio::io::AsyncWriteExt::write_all(&mut audio, &chunk).await?;
}
tokio::io::AsyncWriteExt::flush(&mut audio).await?;
drop(audio);
let text =
fsdy::TrackFile::from_track_with_file(track, Path::new(&audio_name)).to_toml()?;
tokio::fs::write(dir.join(fsdy::track_file_name(index, &track.title)), text).await?;
Ok(())
}
}
/// Captures the subtree at `source_path` as `dir/<name>/`, overwriting an
/// existing folder of that name.
///
/// Validates `name` ([`fsdy::validate_folder_name`], nothing reserved),
/// builds the whole capture in a hidden `.tmp-<name>` sibling via
/// [`write_tree`], and swaps it into place. Every failure removes the temp
/// folder — nothing half-written survives, not even hidden.
pub async fn capture_into<C>(
dir: &Path,
client: &C,
source_path: &str,
name: &str,
caps: Caps,
sink: &Sink,
) -> Result<(), CaptureError>
where
C: ProviderClient + Sync,
{
let name = fsdy::validate_folder_name(name, &[]).map_err(CaptureError::InvalidName)?;
let tmp = dir.join(format!(".tmp-{name}"));
let written = write_tree(client, source_path, &tmp, caps, sink).await;
if let Err(err) = written {
// Every failure path removes the temp folder: nothing
// half-written survives, not even hidden.
let _ = tokio::fs::remove_dir_all(&tmp).await;
return Err(err);
}
let target = dir.join(name);
if tokio::fs::try_exists(&target).await? {
tokio::fs::remove_dir_all(&target).await?;
}
tokio::fs::rename(&tmp, &target).await?;
Ok(())
}
/// Builds the mirrored tree inside `tmp`: iterative pre-order over
/// [`ProviderClient::get_lib_node`] (a deep tree must not overflow the
/// stack), every child node an order-prefixed folder ([`fsdy::dir_name`]),
/// every track handed to `sink` with its listing index. A `source_path`
/// that is a track captures as a folder with one entry. All-or-nothing:
/// any provider, download, or write failure aborts the whole capture.
async fn write_tree<C>(
client: &C,
source_path: &str,
tmp: &Path,
caps: Caps,
sink: &Sink,
) -> Result<(), CaptureError>
where
C: ProviderClient + Sync,
{
// A leftover temp folder from a crashed or racing capture is stale.
if tokio::fs::try_exists(tmp).await? {
tokio::fs::remove_dir_all(tmp).await?;
}
tokio::fs::create_dir_all(tmp).await?;
let mut bytes_left = caps.max_bytes;
// A track source captures as a folder with one entry.
if client.is_track_path(source_path) {
let track = client
.get_metadata_for_track(source_path)
.await
.map_err(|err| CaptureError::BadSource(format!("{source_path}: {err}")))?;
return write_track(sink, client, &track, tmp, 0, &mut bytes_left).await;
}
let mut dirs = 1usize;
let mut tracks = 0usize;
// Iterative pre-order: a deep provider tree must not overflow the
// stack. Each entry pairs a library path with its mirror folder.
let mut worklist: Vec<(String, PathBuf)> = vec![(source_path.to_string(), tmp.to_path_buf())];
while let Some((lib_path, dir)) = worklist.pop() {
let node = client
.get_lib_node(&lib_path)
.await
.map_err(|err| CaptureError::BadSource(format!("{lib_path}: {err}")))?;
for (index, track) in node.tracks.iter().enumerate() {
tracks += 1;
if tracks > caps.max_tracks {
return Err(CaptureError::TooLarge("too many tracks"));
}
write_track(sink, client, track, &dir, index, &mut bytes_left).await?;
}
for (index, child) in node.children.iter().enumerate() {
dirs += 1;
if dirs > caps.max_dirs {
return Err(CaptureError::TooLarge("too many directories"));
}
let child_dir = dir.join(fsdy::dir_name(index, &child.title));
tokio::fs::create_dir(&child_dir).await?;
worklist.push((child.path.clone(), child_dir));
}
}
Ok(())
}
/// Writes one visited track into `dir` at listing position `index`,
/// drawing downloaded bytes from `bytes_left`.
async fn write_track<C>(
sink: &Sink,
client: &C,
track: &Track,
dir: &Path,
index: usize,
bytes_left: &mut u64,
) -> Result<(), CaptureError>
where
C: ProviderClient + Sync,
{
match sink {
Sink::Link => {
let text = fsdy::TrackFile::from_track(track).to_toml()?;
let file = dir.join(fsdy::track_file_name(index, &track.title));
tokio::fs::write(file, text).await?;
Ok(())
}
Sink::Download(downloader) => {
downloader
.download_track(client, track, dir, index, bytes_left)
.await
}
}
}
/// Picks the audio file extension: the response `Content-Type` first
/// (`audio/flac` → `flac`, `audio/mp4`/`audio/m4a` → `m4a`, `audio/mpeg` →
/// `mp3`, `audio/ogg` → `ogg`, `audio/wav` → `wav`), then the URL path's
/// extension, then `bin` — the player probes by content, the extension is
/// a hint.
fn extension_for(content_type: Option<&str>, url: &str) -> String {
let mapped = content_type
.and_then(|ct| ct.split(';').next())
.map(|essence| essence.trim().to_ascii_lowercase())
.and_then(|essence| match essence.as_str() {
"audio/flac" | "audio/x-flac" => Some("flac"),
"audio/mp4" | "audio/m4a" | "audio/x-m4a" => Some("m4a"),
"audio/mpeg" | "audio/mp3" => Some("mp3"),
"audio/ogg" => Some("ogg"),
"audio/wav" | "audio/x-wav" => Some("wav"),
_ => None,
});
match mapped.or_else(|| url_extension(url)) {
Some(ext) => ext.to_string(),
None => "bin".to_string(),
}
}
/// The extension of a URL's last path segment (query and fragment
/// stripped), when it looks like one: short and alphanumeric.
fn url_extension(url: &str) -> Option<&str> {
let path = url.split(['?', '#']).next()?;
let segment = path.rsplit('/').next()?;
let (stem, ext) = segment.rsplit_once('.')?;
let plausible = !stem.is_empty()
&& !ext.is_empty()
&& ext.len() <= 5
&& ext.chars().all(|c| c.is_ascii_alphanumeric());
plausible.then_some(ext)
}
/// The audio file name for track `index` titled `title` with `ext`, next
/// to its toml: `NNNN <title>.<ext>` through the same sanitizer as the
/// toml name, so the pair sorts together — and what the toml's relative
/// `file` playable points at.
fn audio_file_name(index: usize, title: &str, ext: &str) -> String {
format!("{}.{ext}", fsdy::dir_name(index, title))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn extension_prefers_content_type_then_url_then_bin() {
for (ct, ext) in [
("audio/flac", "flac"),
("audio/mp4", "m4a"),
("audio/m4a", "m4a"),
("audio/mpeg", "mp3"),
("audio/ogg", "ogg"),
("audio/wav", "wav"),
] {
assert_eq!(extension_for(Some(ct), "https://x.test/s"), ext);
}
// Content-Type parameters must not confuse the mapping.
assert_eq!(
extension_for(Some("audio/flac; charset=binary"), "https://x.test/s"),
"flac"
);
// Unknown or missing types fall back to the URL path's extension…
assert_eq!(
extension_for(None, "https://x.test/media/track.m4a?token=abc"),
"m4a"
);
assert_eq!(
extension_for(Some("application/octet-stream"), "https://x.test/a.flac"),
"flac"
);
// …and to `bin` when the URL has none either.
assert_eq!(extension_for(None, "https://x.test/stream"), "bin");
assert_eq!(extension_for(None, "not a url"), "bin");
}
#[test]
fn audio_files_pair_with_their_toml_names() {
// Same prefix and sanitized stem as `fsdy::track_file_name`, so the
// audio file sorts right next to its toml.
assert_eq!(
audio_file_name(0, "We Will Rock You", "flac"),
"0001 We Will Rock You.flac"
);
assert_eq!(audio_file_name(11, "a/b", "mp3"), "0012 a_b.mp3");
}
}

View File

@ -0,0 +1,442 @@
//! Downloaded library subtrees ("captures") on disk
//! (see `architecture/captures.md`).
//!
//! Every capture is a folder under `<config>/crabidy/captures/` that
//! mirrors the captured subtree like a bookmark, except each track's audio
//! is **downloaded** next to its order-prefixed `*.cbd-track.toml`, and the
//! toml's playable is a relative `file` pointing at it — replaying a
//! capture needs no provider round trip. The same directory is mounted
//! read-only into the library as `/captures` by an `fsdy` instance (with
//! editable top-level folders); this module is the only writer.
use std::path::{Path, PathBuf};
use crabidy_core::ProviderClient;
use crate::capture::{Caps, CaptureError};
/// The library mount point of the captures directory.
pub const CAPTURES_PROVIDER_ROOT: &str = "/captures";
/// The captures directory: `captures/` inside the crabidy config
/// directory. `None` when the platform has no config directory.
pub fn captures_dir() -> Option<PathBuf> {
dirs::config_dir().map(|d| d.join("crabidy").join("captures"))
}
/// Writes download captures. All audio is fetched through one shared HTTP
/// client; the whole capture is built as a hidden temp sibling and swapped
/// into place, so a crash or failed download never leaves a half-written
/// capture next to intact ones.
#[derive(Debug)]
pub struct CaptureStore {
dir: PathBuf,
sink: crate::capture::Sink,
}
impl CaptureStore {
/// Opens the store at `dir`, creating the directory (and parents) if
/// missing, and builds the shared HTTP client.
pub async fn open(dir: PathBuf) -> Result<Self, CaptureError> {
tokio::fs::create_dir_all(&dir).await?;
let downloader = crate::capture::Downloader::new()
.map_err(|err| CaptureError::Download(format!("cannot build http client: {err}")))?;
Ok(Self {
dir,
sink: crate::capture::Sink::Download(downloader),
})
}
/// The store directory (what the `/captures` provider instance
/// mounts).
pub fn dir(&self) -> &Path {
&self.dir
}
/// Captures the subtree at `source_path` as the capture `name`,
/// downloading every track's audio; overwrites an existing capture of
/// that name.
///
/// The capture **root** must opt in (`architecture/captures.md` D4): a
/// directory source must report `is_downloadable`, a track source's
/// parent node must — otherwise [`CaptureError::Unsupported`]. The
/// walk, caps ([`crate::capture::DOWNLOAD_CAPS`]), tmp-and-swap, and
/// all-or-nothing cleanup are [`crate::capture::capture_into`]'s.
pub async fn capture<C>(
&self,
client: &C,
source_path: &str,
name: &str,
) -> Result<(), CaptureError>
where
C: ProviderClient + Sync,
{
self.capture_with_caps(client, source_path, name, crate::capture::DOWNLOAD_CAPS)
.await
}
/// [`Self::capture`] with explicit caps — the seam the cap tests use.
async fn capture_with_caps<C>(
&self,
client: &C,
source_path: &str,
name: &str,
caps: Caps,
) -> Result<(), CaptureError>
where
C: ProviderClient + Sync,
{
source_allows_download(client, source_path).await?;
crate::capture::capture_into(&self.dir, client, source_path, name, caps, &self.sink).await
}
}
/// Checks that the capture root allows downloads: the node at
/// `source_path` (or, for a track path, its parent node) must set
/// `is_downloadable`. An unreadable root is [`CaptureError::BadSource`].
async fn source_allows_download<C>(client: &C, source_path: &str) -> Result<(), CaptureError>
where
C: ProviderClient + Sync,
{
let node_path = if client.is_track_path(source_path) {
crabidy_core::parent_path(source_path).unwrap_or(source_path)
} else {
source_path
};
let node = client
.get_lib_node(node_path)
.await
.map_err(|err| CaptureError::BadSource(format!("{node_path}: {err}")))?;
if node.is_downloadable {
Ok(())
} else {
Err(CaptureError::Unsupported)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::capture::{Caps, DOWNLOAD_CAPS};
use async_trait::async_trait;
use crabidy_core::proto::crabidy::{LibraryNode, LibraryNodeChild, Track};
use crabidy_core::ProviderError;
use std::collections::HashMap;
use std::fs;
use std::path::Path;
use tempfile::TempDir;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
/// Serves every request with one fixed response and returns a URL for
/// it. Minimal HTTP/1.1 on a loopback socket — enough for reqwest.
async fn serve(status: &'static str, content_type: &'static str, body: Vec<u8>) -> String {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind test server");
let addr = listener.local_addr().expect("test server addr");
tokio::spawn(async move {
while let Ok((mut sock, _)) = listener.accept().await {
let body = body.clone();
tokio::spawn(async move {
// Read until the header terminator; the request itself
// is irrelevant.
let mut buf = Vec::new();
let mut chunk = [0u8; 1024];
loop {
match sock.read(&mut chunk).await {
Ok(0) | Err(_) => break,
Ok(n) => {
buf.extend_from_slice(&chunk[..n]);
if buf.windows(4).any(|w| w == b"\r\n\r\n") {
break;
}
}
}
}
let head = format!(
"HTTP/1.1 {status}\r\nContent-Type: {content_type}\r\n\
Content-Length: {}\r\nConnection: close\r\n\r\n",
body.len()
);
let _ = sock.write_all(head.as_bytes()).await;
let _ = sock.write_all(&body).await;
let _ = sock.shutdown().await;
});
}
});
format!("http://{addr}/stream")
}
/// A provider with one album (`/mock/a`: tracks `one`, `two`) whose
/// stream URLs are the test server's, and a node-level download
/// blessing toggle.
#[derive(Debug)]
struct MockProvider {
urls: HashMap<String, String>,
downloadable: bool,
}
impl MockProvider {
fn new(urls: &[(&str, &str)], downloadable: bool) -> Self {
Self {
urls: urls
.iter()
.map(|(p, u)| (p.to_string(), u.to_string()))
.collect(),
downloadable,
}
}
fn track(&self, path: &str) -> Track {
let title = match path {
"/mock/a/1" => "one",
"/mock/a/2" => "two",
other => other,
};
Track {
path: path.to_string(),
artist: "mock".to_string(),
title: title.to_string(),
duration: Some(10),
album: None,
}
}
}
#[async_trait]
impl ProviderClient for MockProvider {
async fn init(_s: &str) -> Result<Self, ProviderError> {
Err(ProviderError::NotSupported)
}
fn settings(&self) -> String {
String::new()
}
fn is_track_path(&self, path: &str) -> bool {
path.starts_with("/mock/a/") && path.len() > "/mock/a/".len()
}
async fn get_urls_for_track(&self, track_path: &str) -> Result<Vec<String>, ProviderError> {
self.urls
.get(track_path)
.map(|u| vec![u.clone()])
.ok_or(ProviderError::FetchError)
}
async fn get_metadata_for_track(&self, track_path: &str) -> Result<Track, ProviderError> {
if !self.is_track_path(track_path) {
return Err(ProviderError::MalformedPath);
}
Ok(self.track(track_path))
}
fn get_lib_root(&self) -> LibraryNode {
LibraryNode::new()
}
async fn get_lib_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
let mut node = LibraryNode::new();
node.path = path.to_string();
node.is_downloadable = self.downloadable;
match path {
"/mock" => {
node.title = "mock".to_string();
node.children = vec![LibraryNodeChild {
is_downloadable: self.downloadable,
..LibraryNodeChild::new("/mock/a".to_string(), "a".to_string(), true)
}];
}
"/mock/a" => {
node.title = "a".to_string();
node.is_queable = true;
node.tracks = vec![self.track("/mock/a/1"), self.track("/mock/a/2")];
}
_ => return Err(ProviderError::MalformedPath),
}
Ok(node)
}
async fn create_lib_node(
&self,
_parent_path: &str,
_title: &str,
) -> Result<LibraryNode, ProviderError> {
Err(ProviderError::NotSupported)
}
async fn rename_lib_node(
&self,
_path: &str,
_new_title: &str,
) -> Result<LibraryNode, ProviderError> {
Err(ProviderError::NotSupported)
}
async fn delete_lib_node(&self, _path: &str) -> Result<LibraryNode, ProviderError> {
Err(ProviderError::NotSupported)
}
}
async fn store() -> (CaptureStore, TempDir) {
let dir = TempDir::new().expect("store tempdir");
let store = CaptureStore::open(dir.path().join("captures"))
.await
.expect("open creates the directory");
(store, dir)
}
fn visible(dir: &Path) -> Vec<String> {
let mut names: Vec<String> = fs::read_dir(dir)
.expect("dir")
.map(|e| e.expect("entry").file_name().to_string_lossy().into_owned())
.filter(|n| !n.starts_with('.'))
.collect();
names.sort_by_key(|n| n.to_lowercase());
names
}
#[tokio::test]
async fn download_capture_writes_audio_next_to_pointing_tomls() {
let url = serve("200 OK", "audio/flac", b"flacbytes".to_vec()).await;
let mock = MockProvider::new(&[("/mock/a/1", &url), ("/mock/a/2", &url)], true);
let (store, _dir) = store().await;
store
.capture(&mock, "/mock/a", "faves")
.await
.expect("capture");
let root = store.dir().join("faves");
assert_eq!(
visible(&root),
vec![
"0001 one.cbd-track.toml".to_string(),
"0001 one.flac".into(),
"0002 two.cbd-track.toml".into(),
"0002 two.flac".into(),
]
);
assert_eq!(
fs::read(root.join("0001 one.flac")).expect("audio"),
b"flacbytes"
);
// The toml points at its sibling with a *relative* file playable.
let toml = fs::read_to_string(root.join("0002 two.cbd-track.toml")).expect("toml");
let file = fsdy::TrackFile::parse(&toml).expect("parses");
assert_eq!(
file.playable().expect("playable"),
fsdy::Playable::File("0002 two.flac".into())
);
assert_eq!(file.title, "two");
// Replay: the capture resolves through a /captures instance and the
// audio resolves to the absolute sibling path — no provider round
// trip left.
let captures = fsdy::Client::new(CAPTURES_PROVIDER_ROOT, store.dir().to_path_buf())
.expect("captures instance");
let (chunk_tx, chunk_rx) = flume::bounded(8);
captures
.resolve_tracks_into("/captures/faves", chunk_tx)
.await
.expect("resolve");
let tracks: Vec<Track> = chunk_rx.into_iter().flatten().collect();
assert_eq!(tracks.len(), 2);
let urls = captures
.get_urls_for_track(&tracks[0].path)
.await
.expect("urls");
assert_eq!(urls, vec![root.join("0001 one.flac").display().to_string()]);
}
#[tokio::test]
async fn capture_requires_the_root_download_blessing() {
let url = serve("200 OK", "audio/flac", b"x".to_vec()).await;
let mock = MockProvider::new(&[("/mock/a/1", &url), ("/mock/a/2", &url)], false);
let (store, _dir) = store().await;
assert!(matches!(
store.capture(&mock, "/mock/a", "faves").await,
Err(CaptureError::Unsupported)
));
assert!(matches!(
store.capture(&mock, "/mock/a/1", "faves").await,
Err(CaptureError::Unsupported)
));
assert_eq!(fs::read_dir(store.dir()).expect("store dir").count(), 0);
}
#[tokio::test]
async fn capturing_a_single_blessed_track_writes_one_pair() {
let url = serve("200 OK", "audio/mpeg", b"mp3bytes".to_vec()).await;
let mock = MockProvider::new(&[("/mock/a/1", &url)], true);
let (store, _dir) = store().await;
store
.capture(&mock, "/mock/a/1", "just one")
.await
.expect("capture track");
assert_eq!(
visible(&store.dir().join("just one")),
vec!["0001 one.cbd-track.toml".to_string(), "0001 one.mp3".into()]
);
}
#[tokio::test]
async fn download_capture_is_all_or_nothing() {
let ok = serve("200 OK", "audio/flac", b"x".to_vec()).await;
let gone = serve("404 Not Found", "text/plain", Vec::new()).await;
let mock = MockProvider::new(&[("/mock/a/1", &ok), ("/mock/a/2", &gone)], true);
let (store, _dir) = store().await;
let err = store
.capture(&mock, "/mock/a", "faves")
.await
.expect_err("a failed download aborts the capture");
assert!(matches!(err, CaptureError::Download(_)), "got {err:?}");
// Nothing half-written survives, not even hidden temp folders.
assert_eq!(fs::read_dir(store.dir()).expect("store dir").count(), 0);
}
#[tokio::test]
async fn download_capture_enforces_its_caps() {
let url = serve("200 OK", "audio/flac", b"0123456789".to_vec()).await;
let mock = MockProvider::new(&[("/mock/a/1", &url), ("/mock/a/2", &url)], true);
let (store, _dir) = store().await;
// Byte budget: two 10-byte bodies against a 15-byte budget.
let caps = Caps {
max_bytes: 15,
..DOWNLOAD_CAPS
};
let err = store
.capture_with_caps(&mock, "/mock/a", "big", caps)
.await
.expect_err("over the byte budget");
assert!(matches!(err, CaptureError::TooLarge(_)), "got {err:?}");
// Track cap.
let caps = Caps {
max_tracks: 1,
..DOWNLOAD_CAPS
};
let err = store
.capture_with_caps(&mock, "/mock/a", "big", caps)
.await
.expect_err("over the track cap");
assert!(matches!(err, CaptureError::TooLarge(_)), "got {err:?}");
assert_eq!(fs::read_dir(store.dir()).expect("store dir").count(), 0);
}
#[tokio::test]
async fn capture_validates_names_and_overwrites() {
let url = serve("200 OK", "audio/flac", b"x".to_vec()).await;
let mock = MockProvider::new(&[("/mock/a/1", &url), ("/mock/a/2", &url)], true);
let (store, _dir) = store().await;
assert!(matches!(
store.capture(&mock, "/mock/a", "a/b").await,
Err(CaptureError::InvalidName(_))
));
store
.capture(&mock, "/mock/a", "faves")
.await
.expect("first capture");
store
.capture(&mock, "/mock/a/1", "faves")
.await
.expect("overwrite");
// The overwrite fully replaces the older, larger capture.
assert_eq!(
visible(&store.dir().join("faves")),
vec![
"0001 one.cbd-track.toml".to_string(),
"0001 one.flac".into()
]
);
}
}

View File

@ -1,4 +1,6 @@
pub mod bookmark_store; pub mod bookmark_store;
pub mod capture;
pub mod capture_store;
pub mod queue_store; pub mod queue_store;
use crabidy_core::proto::crabidy::{Queue, Track}; use crabidy_core::proto::crabidy::{Queue, Track};

View File

@ -205,12 +205,16 @@ pub enum ProviderCommand {
result_tx: flume::Sender<Result<LibraryNode, ProviderError>>, result_tx: flume::Sender<Result<LibraryNode, ProviderError>>,
}, },
/// Captures the queueable subtree at `path` as the bookmark `name` /// Captures the queueable subtree at `path` as the bookmark `name`
/// (see `architecture/bookmarks.md` D1D3). Handled on a spawned task — /// (see `architecture/bookmarks.md` D1D3) — or, with `download`, as
/// a large walk must not block the orchestrator loop. /// the download capture `name` under `/captures`, fetching every
/// track's audio (see `architecture/captures.md`). Handled on a
/// spawned task — a large walk or download must not block the
/// orchestrator loop.
CaptureLibraryNode { CaptureLibraryNode {
path: String, path: String,
name: String, name: String,
result_tx: flume::Sender<Result<(), crabidy_server::bookmark_store::CaptureError>>, download: bool,
result_tx: flume::Sender<Result<(), crabidy_server::capture::CaptureError>>,
}, },
} }

View File

@ -5,6 +5,7 @@ use crabidy_core::{
ProviderClient, ProviderError, ProviderClient, ProviderError,
}; };
use crabidy_server::bookmark_store::{BookmarkStore, BOOKMARKS_PROVIDER_ROOT}; use crabidy_server::bookmark_store::{BookmarkStore, BOOKMARKS_PROVIDER_ROOT};
use crabidy_server::capture_store::{CaptureStore, CAPTURES_PROVIDER_ROOT};
use crabidy_server::queue_store::{CURRENT_QUEUE_NAME, QUEUES_PROVIDER_ROOT}; use crabidy_server::queue_store::{CURRENT_QUEUE_NAME, QUEUES_PROVIDER_ROOT};
use std::{fs, path::PathBuf, sync::Arc}; use std::{fs, path::PathBuf, sync::Arc};
use tracing::{debug, debug_span, error, instrument, warn, Instrument}; use tracing::{debug, debug_span, error, instrument, warn, Instrument};
@ -25,9 +26,16 @@ pub struct ProviderOrchestrator {
/// `/bookmarks` (architecture/bookmarks.md D1). `None` without a /// `/bookmarks` (architecture/bookmarks.md D1). `None` without a
/// config directory. /// config directory.
bookmarks_client: Option<Arc<fsdy::Client>>, bookmarks_client: Option<Arc<fsdy::Client>>,
/// The capture writer; `None` disables `CaptureLibraryNode` (and the /// The bookmark writer; `None` disables link captures (and the
/// `/bookmarks` mount goes with it). /// `/bookmarks` mount goes with it).
bookmark_store: Option<Arc<BookmarkStore>>, bookmark_store: Option<Arc<BookmarkStore>>,
/// Fourth `fsdy` instance over the captures folder, mounted at
/// `/captures` (architecture/captures.md D1). `None` without a config
/// directory.
captures_client: Option<Arc<fsdy::Client>>,
/// The download-capture writer; `None` disables download captures
/// (and the `/captures` mount goes with it).
capture_store: Option<Arc<CaptureStore>>,
} }
/// Whether a path belongs to the filesystem provider. /// Whether a path belongs to the filesystem provider.
@ -46,6 +54,11 @@ fn bookmarks_owns(path: &str) -> bool {
path == BOOKMARKS_PROVIDER_ROOT || path.starts_with("/bookmarks/") path == BOOKMARKS_PROVIDER_ROOT || path.starts_with("/bookmarks/")
} }
/// Whether a path belongs to the captures provider instance.
fn captures_owns(path: &str) -> bool {
path == CAPTURES_PROVIDER_ROOT || path.starts_with("/captures/")
}
impl ProviderOrchestrator { impl ProviderOrchestrator {
/// The fs client, or `MalformedPath` (with a warning) when the /// The fs client, or `MalformedPath` (with a warning) when the
/// provider is disabled — a `/fs` path then has no owner. /// provider is disabled — a `/fs` path then has no owner.
@ -73,6 +86,15 @@ impl ProviderOrchestrator {
ProviderError::MalformedPath ProviderError::MalformedPath
}) })
} }
/// The captures client, or `MalformedPath` (with a warning) when the
/// instance is disabled — a `/captures` path then has no owner.
fn captures_provider(&self) -> Result<&fsdy::Client, ProviderError> {
self.captures_client.as_deref().ok_or_else(|| {
warn!("captures library is disabled");
ProviderError::MalformedPath
})
}
pub fn run(self) { pub fn run(self) {
tokio::spawn(async move { tokio::spawn(async move {
// Behind an Arc so long-running resolves can be spawned onto // Behind an Arc so long-running resolves can be spawned onto
@ -148,20 +170,29 @@ impl ProviderOrchestrator {
ProviderCommand::CaptureLibraryNode { ProviderCommand::CaptureLibraryNode {
path, path,
name, name,
download,
result_tx, result_tx,
} => { } => {
// Spawned: capturing a large artist walks many provider // Spawned: capturing a large artist walks many provider
// nodes and must not block this loop (the walk itself // nodes (and, for downloads, streams audio) and must not
// calls back into `get_lib_node` via `this`). // block this loop (the walk itself calls back into
// `get_lib_node` via `this`).
let this = Arc::clone(&self); let this = Arc::clone(&self);
tokio::spawn( tokio::spawn(
async move { async move {
let result = match &this.bookmark_store { let result = if download {
Some(store) => store.capture(&*this, &path, &name).await, match &this.capture_store {
None => Err(crabidy_server::bookmark_store::CaptureError::Disabled), Some(store) => store.capture(&*this, &path, &name).await,
None => Err(crabidy_server::capture::CaptureError::Disabled),
}
} else {
match &this.bookmark_store {
Some(store) => store.capture(&*this, &path, &name).await,
None => Err(crabidy_server::capture::CaptureError::Disabled),
}
}; };
if let Err(err) = &result { if let Err(err) = &result {
warn!(path, name, "cannot capture subtree: {err}"); warn!(path, name, download, "cannot capture subtree: {err}");
} }
if let Err(err) = result_tx.send_async(result).await { if let Err(err) = result_tx.send_async(result).await {
error!("failed to send capture_library_node result: {err}"); error!("failed to send capture_library_node result: {err}");
@ -263,6 +294,30 @@ impl ProviderClient for ProviderOrchestrator {
} }
} }
}); });
// Captures: like bookmarks, but the store downloads every track's
// audio next to its toml (architecture/captures.md D1). Non-fatal.
let capture_store = match crabidy_server::capture_store::captures_dir() {
Some(dir) => match CaptureStore::open(dir).await {
Ok(store) => Some(Arc::new(store)),
Err(err) => {
warn!("captures disabled: {err}");
None
}
},
None => {
warn!("captures disabled: no config directory");
None
}
};
let captures_client = capture_store.as_ref().and_then(|store| {
match fsdy::Client::new(CAPTURES_PROVIDER_ROOT, store.dir().to_path_buf()) {
Ok(client) => Some(Arc::new(client.with_editable_top_level(&[]))),
Err(err) => {
warn!("captures library disabled: {err}");
None
}
}
});
let (provider_tx, provider_rx) = flume::bounded(100); let (provider_tx, provider_rx) = flume::bounded(100);
Ok(Self { Ok(Self {
provider_rx, provider_rx,
@ -272,6 +327,8 @@ impl ProviderClient for ProviderOrchestrator {
queues_client, queues_client,
bookmarks_client, bookmarks_client,
bookmark_store, bookmark_store,
captures_client,
capture_store,
}) })
} }
@ -302,6 +359,12 @@ impl ProviderClient for ProviderOrchestrator {
.as_ref() .as_ref()
.is_some_and(|bookmarks| bookmarks.is_track_path(path)); .is_some_and(|bookmarks| bookmarks.is_track_path(path));
} }
if captures_owns(path) {
return self
.captures_client
.as_ref()
.is_some_and(|captures| captures.is_track_path(path));
}
false false
} }
@ -322,6 +385,12 @@ impl ProviderClient for ProviderOrchestrator {
.get_urls_for_track(track_path) .get_urls_for_track(track_path)
.await; .await;
} }
if captures_owns(track_path) {
return self
.captures_provider()?
.get_urls_for_track(track_path)
.await;
}
warn!(path = track_path, "no provider owns this track path"); warn!(path = track_path, "no provider owns this track path");
Err(ProviderError::MalformedPath) Err(ProviderError::MalformedPath)
} }
@ -346,6 +415,12 @@ impl ProviderClient for ProviderOrchestrator {
.get_metadata_for_track(track_path) .get_metadata_for_track(track_path)
.await; .await;
} }
if captures_owns(track_path) {
return self
.captures_provider()?
.get_metadata_for_track(track_path)
.await;
}
warn!(path = track_path, "no provider owns this track path"); warn!(path = track_path, "no provider owns this track path");
Err(ProviderError::MalformedPath) Err(ProviderError::MalformedPath)
} }
@ -373,6 +448,14 @@ impl ProviderClient for ProviderOrchestrator {
); );
root_node.children.push(child); root_node.children.push(child);
} }
if self.captures_client.is_some() {
let child = LibraryNodeChild::new(
CAPTURES_PROVIDER_ROOT.to_owned(),
"captures".to_owned(),
false,
);
root_node.children.push(child);
}
root_node root_node
} }
@ -394,6 +477,9 @@ impl ProviderClient for ProviderOrchestrator {
if bookmarks_owns(path) { if bookmarks_owns(path) {
return self.bookmarks_provider()?.get_lib_node(path).await; return self.bookmarks_provider()?.get_lib_node(path).await;
} }
if captures_owns(path) {
return self.captures_provider()?.get_lib_node(path).await;
}
warn!(path, "no provider owns this path"); warn!(path, "no provider owns this path");
Err(ProviderError::MalformedPath) Err(ProviderError::MalformedPath)
} }
@ -427,6 +513,12 @@ impl ProviderClient for ProviderOrchestrator {
.create_lib_node(parent_path, title) .create_lib_node(parent_path, title)
.await; .await;
} }
if captures_owns(parent_path) {
return self
.captures_provider()?
.create_lib_node(parent_path, title)
.await;
}
warn!(parent_path, "no provider supports creating nodes here"); warn!(parent_path, "no provider supports creating nodes here");
Err(ProviderError::NotSupported) Err(ProviderError::NotSupported)
} }
@ -457,6 +549,12 @@ impl ProviderClient for ProviderOrchestrator {
.rename_lib_node(path, new_title) .rename_lib_node(path, new_title)
.await; .await;
} }
if captures_owns(path) {
return self
.captures_provider()?
.rename_lib_node(path, new_title)
.await;
}
warn!(path, "no provider supports renaming this node"); warn!(path, "no provider supports renaming this node");
Err(ProviderError::NotSupported) Err(ProviderError::NotSupported)
} }
@ -490,6 +588,12 @@ impl ProviderClient for ProviderOrchestrator {
.resolve_tracks_into(path, chunk_tx) .resolve_tracks_into(path, chunk_tx)
.await; .await;
} }
if captures_owns(path) {
return self
.captures_provider()?
.resolve_tracks_into(path, chunk_tx)
.await;
}
warn!(path, "no provider owns this path"); warn!(path, "no provider owns this path");
Err(ProviderError::MalformedPath) Err(ProviderError::MalformedPath)
} }
@ -510,6 +614,9 @@ impl ProviderClient for ProviderOrchestrator {
if bookmarks_owns(path) { if bookmarks_owns(path) {
return self.bookmarks_provider()?.delete_lib_node(path).await; return self.bookmarks_provider()?.delete_lib_node(path).await;
} }
if captures_owns(path) {
return self.captures_provider()?.delete_lib_node(path).await;
}
warn!(path, "no provider supports deleting this node"); warn!(path, "no provider supports deleting this node");
Err(ProviderError::NotSupported) Err(ProviderError::NotSupported)
} }

View File

@ -377,26 +377,33 @@ impl CrabidyService for RpcService {
} }
/// Captures a queueable subtree as a bookmark via the provider loop /// Captures a queueable subtree as a bookmark via the provider loop
/// (structure-preserving snapshot under `/bookmarks/<name>`). /// (structure-preserving snapshot under `/bookmarks/<name>`), or —
/// with `download` — as a download capture under `/captures/<name>`.
/// ///
/// Error mapping is part of the contract: invalid name or /// Error mapping is part of the contract: invalid name or
/// uncapturable source → `invalid_argument`; an over-cap subtree or /// uncapturable source → `invalid_argument`; an over-cap subtree,
/// disabled bookmarks → `failed_precondition`; walk/write failures → /// disabled store, or a source that does not allow downloads →
/// `internal`. /// `failed_precondition`; walk/write/download failures → `internal`.
#[instrument(skip(self, request), fields(path, name))] #[instrument(skip(self, request), fields(path, name, download))]
async fn capture_library_node( async fn capture_library_node(
&self, &self,
request: Request<CaptureLibraryNodeRequest>, request: Request<CaptureLibraryNodeRequest>,
) -> Result<Response<CaptureLibraryNodeResponse>, Status> { ) -> Result<Response<CaptureLibraryNodeResponse>, Status> {
let CaptureLibraryNodeRequest { path, name } = request.into_inner(); let CaptureLibraryNodeRequest {
path,
name,
download,
} = request.into_inner();
tracing::Span::current().record("path", path.as_str()); tracing::Span::current().record("path", path.as_str());
tracing::Span::current().record("name", name.as_str()); tracing::Span::current().record("name", name.as_str());
tracing::Span::current().record("download", download);
debug!("received capture_library_node request"); debug!("received capture_library_node request");
let (result_tx, result_rx) = flume::bounded(1); let (result_tx, result_rx) = flume::bounded(1);
self.provider_tx self.provider_tx
.send_async(ProviderMessage::new(ProviderCommand::CaptureLibraryNode { .send_async(ProviderMessage::new(ProviderCommand::CaptureLibraryNode {
path, path,
name, name,
download,
result_tx, result_tx,
})) }))
.await .await
@ -413,9 +420,11 @@ impl CrabidyService for RpcService {
Err(err @ (CaptureError::InvalidName(_) | CaptureError::BadSource(_))) => { Err(err @ (CaptureError::InvalidName(_) | CaptureError::BadSource(_))) => {
Err(Status::invalid_argument(err.to_string())) Err(Status::invalid_argument(err.to_string()))
} }
Err(err @ (CaptureError::TooLarge(_) | CaptureError::Disabled)) => { Err(
Err(Status::failed_precondition(err.to_string())) err @ (CaptureError::TooLarge(_)
} | CaptureError::Disabled
| CaptureError::Unsupported),
) => Err(Status::failed_precondition(err.to_string())),
Err(err) => { Err(err) => {
error!("capture_library_node failed: {err}"); error!("capture_library_node failed: {err}");
Err(Status::internal("cannot capture the subtree")) Err(Status::internal("cannot capture the subtree"))

View File

@ -201,6 +201,22 @@ impl TrackFile {
} }
} }
/// Like [`Self::from_track`], but the playable is a local
/// [`Playable::File`] at `file` instead of a link — used by download
/// captures, where the audio sits next to the toml and `file` is its
/// relative name (see `architecture/captures.md` D3). Relative paths
/// resolve against the track file's directory at play time, so the
/// capture folder stays relocatable.
pub fn from_track_with_file(track: &Track, file: &Path) -> Self {
let mut this = Self::from_track(track);
this.playable = PlayableSpec {
file: Some(file.to_path_buf()),
url: None,
link: None,
};
this
}
/// Serializes this file to TOML text. /// Serializes this file to TOML text.
/// ///
/// Only fails when TOML cannot represent the value /// Only fails when TOML cannot represent the value
@ -482,6 +498,7 @@ impl Client {
tracks, tracks,
is_queable: true, is_queable: true,
is_creatable: false, is_creatable: false,
is_downloadable: false,
}) })
} }
@ -629,6 +646,7 @@ impl ProviderClient for Client {
tracks: Vec::new(), tracks: Vec::new(),
is_queable: true, is_queable: true,
is_creatable: false, is_creatable: false,
is_downloadable: false,
} }
} }
@ -1158,6 +1176,51 @@ mod tests {
assert_eq!(restored, track); assert_eq!(restored, track);
} }
#[tokio::test]
async fn from_track_with_file_plays_the_relative_sibling() {
let track = Track {
path: "/tidal/artists/1/2/3".to_string(),
artist: "Queen".to_string(),
title: "One".to_string(),
duration: Some(90),
album: Some(Album {
title: "Greatest".to_string(),
release_date: None,
}),
};
let file = TrackFile::from_track_with_file(&track, Path::new("0001 One.flac"));
let toml_text = file.to_toml().expect("serialize");
let reparsed = TrackFile::parse(&toml_text).expect("reparse");
assert_eq!(
reparsed.playable().expect("playable"),
Playable::File("0001 One.flac".into())
);
// File playables keep the *library* path (no link rewrite), and the
// metadata survives the round trip.
let lib_path = "/captures/faves/0001%20One.cbd-track.toml";
let listed = reparsed.to_track(lib_path);
assert_eq!(listed.path, lib_path);
assert_eq!(listed.title, track.title);
assert_eq!(listed.album, track.album);
// End to end: a capture folder replays through an instance and the
// audio resolves to the absolute sibling path.
let dir = TempDir::new().expect("tempdir");
let faves = dir.path().join("faves");
fs::create_dir_all(&faves).expect("mkdir");
fs::write(faves.join("0001 One.flac"), b"flac").expect("audio");
fs::write(faves.join(track_file_name(0, "One")), toml_text).expect("toml");
let client = Client::new("/captures", dir.path().to_path_buf()).expect("instance");
let urls = client
.get_urls_for_track("/captures/faves/0001%20One.cbd-track.toml")
.await
.expect("resolve");
assert_eq!(
urls,
vec![faves.join("0001 One.flac").display().to_string()]
);
}
#[test] #[test]
fn from_track_serializes_sparse_metadata() { fn from_track_serializes_sparse_metadata() {
// No artist/duration/album: serialization must not fail on `None` // No artist/duration/album: serialization must not fail on `None`

54
plan/captures.md Normal file
View File

@ -0,0 +1,54 @@
# Plan: captures
Ordered tasks; each names its verification (tests in `fsdy/src/lib.rs`,
`crabidy-server/src/capture.rs`, `crabidy-server/src/capture_store.rs`
and/or gates in `quality/captures.md`). Stubs, the proto field, tidal's
flag rule, and the full orchestrator/TUI plumbing exist; the new tests
fail on `todo!()` at plan time.
- [x] **T1 — fsdy file-playable constructor.**
`TrackFile::from_track_with_file`: metadata like `from_track`, playable
`file = <relative sibling>`. Verifies:
`from_track_with_file_plays_the_relative_sibling`.
- [x] **T2 — capture.rs naming + extension helpers.** `audio_file_name`
(shared `ordered_name` semantics via a public fsdy seam) and
`extension_for` (Content-Type map, URL-path fallback, `bin`).
Verifies: `extension_prefers_content_type_then_url_then_bin`,
`audio_files_pair_with_their_toml_names`.
- [x] **T3 — shared walk.** Move the bookmark walk into
`capture::capture_into`/`write_tree` parameterized by `Caps` and
`Sink`; per-track writes go through `write_track` (`Sink::Link` =
today's link file). Rewire `BookmarkStore` onto it (delete its copy;
keep its public API and `BOOKMARK_CAPS`-equivalent behavior).
Verifies: all existing `bookmark_store::tests` unchanged; gates
"Shared walk".
- [x] **T4 — download sink.** `Downloader::new` (connect timeout),
`Sink::Download` in `write_track`: per-track deadline around URL
fetch, GET, and the streamed body against `bytes_left`,
`error_for_status`, extension from the response, audio first then toml
(`from_track_with_file`). Verifies:
`download_capture_writes_audio_next_to_pointing_tomls`,
`download_capture_is_all_or_nothing`,
`download_capture_enforces_its_caps`; gates "Download sink".
- [x] **T5 — CaptureStore.** `open` (create dir + build downloader),
`capture` → blessing check (`source_allows_download`: node or track
parent) then `capture_with_caps(DOWNLOAD_CAPS)``capture_into`.
Verifies: `capture_requires_the_root_download_blessing`,
`capturing_a_single_blessed_track_writes_one_pair`,
`capture_validates_names_and_overwrites`; gates "Opt-in".
- [x] **T6 — TUI tests.** Binding test for `W` (Library scope, shift),
`selected_downloadable` gating (queueable-but-not-downloadable stays
closed; tracks inherit the node flag), overlay label `capture`,
submit carries `download: true`. Verifies: new `cbd-tui` tests; gates
"TUI".
- [x] **T7 — full verification.** Whole workspace suite green;
clippy/fmt/taplo/markdownlint clean; walk every gate in
`quality/captures.md` and tick it; no `todo!()` left.
- [x] **T8 — live smoke test.** Capture a real Tidal track with download
through the provider layer into a temp store (a single track keeps the
probe cheap; the multi-track walk is unit-covered), verify the audio
exists with a plausible size, replay through a `/captures` instance,
and play-resolve it — remove any temporary probe afterwards.
- [x] **T9 — docs.** `plan/summary.md` section incl. deviations;
reconcile `architecture/captures.md` (tidal flag rule, anything else
that moved).

View File

@ -1,5 +1,52 @@
# Implementation summaries # Implementation summaries
## captures (2026-07-21)
Built per `plan/captures.md`: `W` (shift) on a downloadable library
selection captures the subtree like a bookmark, but into
`<config>/crabidy/captures/<name>/` with every track's audio
**downloaded** next to its order-prefixed toml — the toml's playable is
the audio file's *relative* name (`TrackFile::from_track_with_file`), so
a capture plays with no provider round trip and the folder stays
relocatable. `/captures` is a fourth `fsdy` instance (editable top
level, nothing reserved): browse, queue, rename (`e`), delete (`d`),
and re-capture to refresh; the audio files are invisible to listings
(only dirs and `*.cbd-track.toml` count).
The bookmark walk was extracted into `capture.rs`
(`capture_into`/`write_tree`, `Caps`, one `CaptureError` for both
stores) parameterized by a per-track `Sink``Link` is byte-identical
bookmark behavior, `Download` fetches the first `get_urls_for_track`
URL through one shared reqwest client (30 s connect timeout, 600 s
per-track deadline, no retries), streams the body to disk against a
capture-wide byte budget, picks the extension from `Content-Type` (URL
path, then `bin`, as fallbacks), and writes the toml only after the
audio succeeded. Download caps: 1 000 dirs, 500 tracks, 4 GiB. Still
all-or-nothing with temp cleanup; downloads are sequential inside the
one spawned capture task. Download error messages carry the track's
library path, never the stream URL (`reqwest::Error::without_url`).
Nodes opt in via new additive proto flags
(`LibraryNode.is_downloadable = 8`, `LibraryNodeChild = 7`). Tidal sets
them centrally at the end of `get_lib_node`: downloadable = queueable
**or lists tracks** (so search-term track results are downloadable even
though the term node isn't queueable); children mirror `is_queable`;
tracks inherit their node's flag in the TUI. The server re-enforces at
the capture root (`Unsupported` → `failed_precondition`); the rpc
gained `CaptureLibraryNodeRequest.download = 3` (additive; old clients
keep bookmarking).
Deviations from the plan/architecture: the naming helper ended up
`audio_file_name` (the path variant was clippy-dead); the tidal flag
rule grew the "or lists tracks" clause (architecture D4 reconciled);
the live probe downloaded a single real track (8.6 MB m4a,
Content-Type-derived extension, replayed through a `/captures`
instance) instead of a whole album — the multi-track walk is
unit-covered and a full album download is needlessly heavy for a smoke
test. 142 workspace tests green (1 new in `fsdy`, 8 in
`capture`/`capture_store`, 3 TUI + 1 extended); every gate in
`quality/captures.md` checked.
## bookmarks (2026-07-21) ## bookmarks (2026-07-21)
Built per `plan/bookmarks.md`: `w` on a queueable library selection now Built per `plan/bookmarks.md`: `w` on a queueable library selection now

84
quality/captures.md Normal file
View File

@ -0,0 +1,84 @@
# Quality gates: captures
Criteria the implementation must satisfy beyond the automatic tests
(`crabidy-server/src/capture.rs`, `crabidy-server/src/capture_store.rs`,
`fsdy/src/lib.rs`, plus the TUI tests added during implementation). Each
gate is pass/fail by reading the code.
## Shared walk (refactor)
- [x] Bookmarks and download captures run through **one** walk
(`capture::capture_into`/`write_tree`); `bookmark_store` keeps no copy
of the worklist, caps, tmp-and-swap, or cleanup logic.
- [x] The refactor is behavior-preserving for bookmarks: every existing
`bookmark_store` test passes unchanged (module path of `CaptureError`
aside).
- [x] The walk stays iterative (worklist), all-or-nothing, and removes the
temp folder on every failure path — including failed downloads and the
byte budget.
## Download sink
- [x] Every external call is bounded: connect timeout on the shared HTTP
client and one per-track deadline covering URL fetch, request, and the
whole body stream. No retries.
- [x] Bodies are **streamed** to disk (never buffered whole) and counted
against the capture's byte budget while streaming; exceeding it is
`TooLarge`, not partial data left behind.
- [x] Non-2xx responses and transport errors are typed
(`CaptureError::Download`) — no panic on any network condition.
- [x] Download log lines carry paths, names, and counts — never stream
URLs (they embed tokens) and never file contents.
- [x] The toml is written only after its audio file succeeded, with a
relative `file` playable naming the sibling
(`TrackFile::from_track_with_file`); the audio name shares the toml's
order prefix and sanitizer (`capture::audio_file_name`).
- [x] Downloads run sequentially inside the one spawned capture task; the
orchestrator loop keeps serving commands during a capture.
## Opt-in (is_downloadable)
- [x] Tidal sets the flag centrally: nodes are downloadable when queueable
or when they list tracks; children mirror `is_queable`. No per-arm
copies to drift.
- [x] Every other provider (fs, queues, bookmarks, captures, orchestrator
root) leaves the flag `false` — a capture can never be built from
another capture's or bookmark's links masquerading as sources.
- [x] The server enforces the blessing at the capture root (directory
source: its own node; track source: its parent node) with
`CaptureError::Unsupported`; an unreadable root is `BadSource`.
## RPC and orchestrator
- [x] `CaptureLibraryNodeRequest.download` is additive: old clients (no
flag) keep getting bookmarks, byte-for-byte.
- [x] Error mapping: `InvalidName`/`BadSource` → `invalid_argument`;
`TooLarge`/`Disabled`/`Unsupported` → `failed_precondition`;
download/walk/write failures → `internal`.
- [x] The orchestrator routes `/captures` in every `ProviderClient` method
(same completeness as `/bookmarks`), and `get_lib_root` lists the
`captures` child only when the instance exists.
- [x] Captures init is non-fatal: no config dir or an unopenable store
disables download captures and the `/captures` mount, never the server
(and never bookmarks).
- [x] `/captures` mounts with an editable top level (no reserved names):
captures are renamable (`e`) and deletable (`d`) like bookmarks.
## TUI
- [x] `W` (shift) is bound in `Scope::Library` with a help description and
passes the bindings-table invariant tests unchanged; `w` behavior is
untouched.
- [x] The capture overlay opens for `W` only when the bare selection is
queueable **and** downloadable; tracks inherit their node's flag;
marks are ignored. Label: `capture` (vs `bookmark`).
- [x] `MessageFromUi::CaptureNode` carries `download`; a failed capture is
logged and never tears down the poll loop.
## Hygiene
- [x] New public items are documented; docs state error/edge behavior.
- [x] `clippy -D warnings`, `fmt`, `taplo`, `markdownlint` clean on the
whole workspace; all tests green.
- [x] `architecture/captures.md` reconciled where the implementation
diverged (e.g. the exact tidal flag rule).

View File

@ -128,6 +128,7 @@ impl crabidy_core::ProviderClient for Client {
], ],
is_queable: false, is_queable: false,
is_creatable: false, is_creatable: false,
is_downloadable: false,
} }
} }
@ -155,6 +156,7 @@ impl crabidy_core::ProviderClient for Client {
children: Vec::new(), children: Vec::new(),
is_queable: false, is_queable: false,
is_creatable: false, is_creatable: false,
is_downloadable: false,
}; };
let user_id = user_id.ok_or(crabidy_core::ProviderError::UnknownUser)?; let user_id = user_id.ok_or(crabidy_core::ProviderError::UnknownUser)?;
for playlist in self for playlist in self
@ -186,6 +188,7 @@ impl crabidy_core::ProviderClient for Client {
children: Vec::new(), children: Vec::new(),
is_queable: true, is_queable: true,
is_creatable: false, is_creatable: false,
is_downloadable: false,
} }
} }
TidalPath::Artists => { TidalPath::Artists => {
@ -197,6 +200,7 @@ impl crabidy_core::ProviderClient for Client {
children: Vec::new(), children: Vec::new(),
is_queable: false, is_queable: false,
is_creatable: false, is_creatable: false,
is_downloadable: false,
}; };
let user_id = user_id.ok_or(crabidy_core::ProviderError::UnknownUser)?; let user_id = user_id.ok_or(crabidy_core::ProviderError::UnknownUser)?;
for artist in self.get_users_artists(&user_id).await? { for artist in self.get_users_artists(&user_id).await? {
@ -231,6 +235,7 @@ impl crabidy_core::ProviderClient for Client {
children, children,
is_queable: true, is_queable: true,
is_creatable: false, is_creatable: false,
is_downloadable: false,
} }
} }
TidalPath::Album { album, .. } => { TidalPath::Album { album, .. } => {
@ -249,6 +254,7 @@ impl crabidy_core::ProviderClient for Client {
children: Vec::new(), children: Vec::new(),
is_queable: true, is_queable: true,
is_creatable: false, is_creatable: false,
is_downloadable: false,
} }
} }
TidalPath::Search => crabidy_core::proto::crabidy::LibraryNode { TidalPath::Search => crabidy_core::proto::crabidy::LibraryNode {
@ -275,6 +281,7 @@ impl crabidy_core::ProviderClient for Client {
.collect(), .collect(),
is_queable: false, is_queable: false,
is_creatable: true, is_creatable: true,
is_downloadable: false,
}, },
TidalPath::SearchTerm(encoded) => { TidalPath::SearchTerm(encoded) => {
let term = crabidy_core::decode_segment(encoded); let term = crabidy_core::decode_segment(encoded);
@ -290,6 +297,16 @@ impl crabidy_core::ProviderClient for Client {
return Err(crabidy_core::ProviderError::MalformedPath); return Err(crabidy_core::ProviderError::MalformedPath);
} }
}; };
// Tidal implements download captures (architecture/captures.md D4):
// every queueable subtree may be captured with `W`, and every node
// that lists tracks blesses them for download (listed tracks
// inherit their node's flag — this covers search-term results,
// whose node is not queueable as a whole).
let mut node = node;
node.is_downloadable = node.is_queable || !node.tracks.is_empty();
for child in &mut node.children {
child.is_downloadable = child.is_queable;
}
Ok(node) Ok(node)
} }
@ -897,6 +914,7 @@ impl Client {
children, children,
is_queable: false, is_queable: false,
is_creatable: false, is_creatable: false,
is_downloadable: false,
}) })
} }