Capture library subtrees with downloaded audio
W on a downloadable library node mirrors the subtree into /captures (a fourth fsdy instance) like a bookmark, but downloads every track's audio next to its toml; the toml points at the sibling by relative name, so captures play with no provider round trip. Nodes opt in via the new is_downloadable flags — Tidal blesses queueable and track-listing nodes. The bookmark walk is now the shared capture walk parameterized by a per-track sink. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
655e8054b8
commit
7f0e900c56
|
|
@ -748,6 +748,7 @@ dependencies = [
|
||||||
"fsdy",
|
"fsdy",
|
||||||
"futures",
|
"futures",
|
||||||
"rand 0.10.2",
|
"rand 0.10.2",
|
||||||
|
"reqwest",
|
||||||
"serde",
|
"serde",
|
||||||
"tempfile",
|
"tempfile",
|
||||||
"thiserror 2.0.19",
|
"thiserror 2.0.19",
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,213 @@
|
||||||
|
# Captures (downloaded subtrees)
|
||||||
|
|
||||||
|
## Context and problem statement
|
||||||
|
|
||||||
|
Bookmarks (`w`) mirror a library subtree as **link** files — replaying a
|
||||||
|
bookmark still needs the original provider. The user wants `W` on a library
|
||||||
|
node to do the same capture into a separate local provider called
|
||||||
|
**captures**, except each track's audio is **downloaded** next to its
|
||||||
|
`.cbd-track.toml`, and the toml points at that file. Playback of a capture
|
||||||
|
then needs no provider round trip at all — it is a fully local copy.
|
||||||
|
|
||||||
|
A library node decides whether it allows `W`; Tidal implements it.
|
||||||
|
|
||||||
|
## Assumptions (confirmed against the code)
|
||||||
|
|
||||||
|
- `PlayableSpec.file` already supports **relative** paths, resolved against
|
||||||
|
the track file's directory at `get_urls_for_track` time (fs-provider D3).
|
||||||
|
A toml next to its audio file can say `file = "0001 Song.flac"` and the
|
||||||
|
whole capture folder stays relocatable (tmp-and-swap, rename, backup).
|
||||||
|
- `fsdy::list_dir` only surfaces directories and `*.cbd-track.toml` files;
|
||||||
|
downloaded audio siblings are invisible to the library listing.
|
||||||
|
- The bookmark walk (`bookmark_store::write_capture`) is an iterative
|
||||||
|
pre-order worklist whose only per-track action is "serialize and write
|
||||||
|
one file" — exactly the seam where a download variant plugs in.
|
||||||
|
- `reqwest` (rustls, `stream`) is already a workspace dependency; tidal
|
||||||
|
stream URLs come from `get_urls_for_track` on the orchestrator, so the
|
||||||
|
download needs no new provider methods.
|
||||||
|
- Uppercase bindings (`K`, `J`) already exist in the TUI bindings table;
|
||||||
|
`W` in `Scope::Library` is free.
|
||||||
|
- `LibraryNode`/`LibraryNodeChild` already model per-node capabilities
|
||||||
|
(`is_queable`, `is_creatable`, …) — the "does this node allow `W`"
|
||||||
|
decision extends that pattern.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
### D1 — Fourth `fsdy` instance at `/captures`
|
||||||
|
|
||||||
|
`<config>/crabidy/captures/` is mounted read-only as `/captures` with an
|
||||||
|
editable top level (no reserved names), exactly like `/bookmarks`. Init is
|
||||||
|
non-fatal: an unopenable store disables `W` and the mount, never the
|
||||||
|
server. Loading a capture is browsing `/captures` and queueing a folder —
|
||||||
|
zero new replay mechanisms.
|
||||||
|
|
||||||
|
### D2 — One shared walk, two track sinks
|
||||||
|
|
||||||
|
Options considered:
|
||||||
|
|
||||||
|
- *(a)* Copy `bookmark_store.rs` and swap the per-track write.
|
||||||
|
- *(b)* Extract the walk into a shared `capture` module parameterized by a
|
||||||
|
**track sink**; bookmarks and captures become thin stores over it.
|
||||||
|
|
||||||
|
**Decision: (b)** — the walk (worklist, caps, temp-and-swap, all-or-nothing
|
||||||
|
cleanup, `BadSource` mapping) is behavior we already tested once and must
|
||||||
|
not fork. `crabidy-server/src/capture.rs` owns `CaptureError`, the caps,
|
||||||
|
and `write_tree(client, source, tmp, caps, sink)`; the sink is an enum
|
||||||
|
(no async-trait indirection):
|
||||||
|
|
||||||
|
- `Sink::Link` — today's bookmark behavior, byte-identical
|
||||||
|
(`TrackFile::from_track`, link playable).
|
||||||
|
- `Sink::Download(Downloader)` — captures (D3).
|
||||||
|
|
||||||
|
`BookmarkStore` keeps its API; `CaptureStore` (in `capture_store.rs`) is
|
||||||
|
its sibling over the captures directory.
|
||||||
|
|
||||||
|
### D3 — Download sink: audio next to the toml, toml points at it
|
||||||
|
|
||||||
|
Per track, in listing order:
|
||||||
|
|
||||||
|
1. `get_urls_for_track` through the orchestrator (any provider that yields
|
||||||
|
URLs works; Tidal is the target). First URL wins.
|
||||||
|
2. HTTP GET via one shared `reqwest` client — connect timeout, one total
|
||||||
|
per-track deadline covering the whole body, **no retries** (a capture
|
||||||
|
is re-runnable and overwrite = refresh; a retry policy can come later).
|
||||||
|
The body is streamed to `NNNN <title>.<ext>` (shared `ordered_name`
|
||||||
|
sanitizer, same 4-digit prefix as the toml so the pair sorts together).
|
||||||
|
3. The extension comes from the response `Content-Type`
|
||||||
|
(`audio/flac` → `flac`, `audio/mp4`/`audio/m4a` → `m4a`,
|
||||||
|
`audio/mpeg` → `mp3`, `audio/ogg` → `ogg`, `audio/wav` → `wav`),
|
||||||
|
falling back to the URL path's extension, then `bin` (the player probes
|
||||||
|
by content; the extension is a hint).
|
||||||
|
4. The toml is written **after** the download succeeds, with
|
||||||
|
`playable.file = "<audio file name>"` (relative, new
|
||||||
|
`TrackFile::from_track_with_file`), keeping metadata identical to a
|
||||||
|
bookmark entry.
|
||||||
|
|
||||||
|
Caps: `MAX_CAPTURE_DIRS` stays 1 000; downloads get their own
|
||||||
|
`MAX_DOWNLOAD_TRACKS = 500` and a total byte budget
|
||||||
|
`MAX_DOWNLOAD_BYTES = 4 GiB` counted while streaming — a runaway artist
|
||||||
|
capture must not fill the disk. Downloads run sequentially (gentle on the
|
||||||
|
provider, trivially bounded memory); the whole capture already runs on a
|
||||||
|
spawned task, so the orchestrator keeps serving.
|
||||||
|
|
||||||
|
All-or-nothing is kept: any failed download aborts the capture and removes
|
||||||
|
the temp folder. A capture that lists 30 tracks *has* 30 playable files.
|
||||||
|
|
||||||
|
### D4 — Nodes opt in via `is_downloadable`
|
||||||
|
|
||||||
|
New proto fields `LibraryNode.is_downloadable = 8` and
|
||||||
|
`LibraryNodeChild.is_downloadable = 7` (additive). Tidal sets the flag
|
||||||
|
centrally at the end of `get_lib_node`: a node is downloadable when it is
|
||||||
|
**queueable or lists tracks** (the "or lists tracks" covers search-term
|
||||||
|
result nodes, which are not queueable as a whole but whose track results
|
||||||
|
are downloadable); children mirror `is_queable`. Everything else (fs,
|
||||||
|
queues, bookmarks, captures, orchestrator roots) leaves the default
|
||||||
|
`false`; capturing a capture is pointless and links must not masquerade
|
||||||
|
as downloads.
|
||||||
|
|
||||||
|
Tracks carry no flag: a listed track inherits its containing node's
|
||||||
|
`is_downloadable` (TUI) — a Tidal album's tracks are downloadable because
|
||||||
|
the album is. The server enforces at the capture **root**: a directory
|
||||||
|
source must report `is_downloadable`, a track source's parent node must
|
||||||
|
(`CaptureError::Unsupported` otherwise). Nested nodes inside the walk are
|
||||||
|
not re-checked — the pressed node's decision governs its subtree.
|
||||||
|
|
||||||
|
### D5 — Wire: the existing rpc gains a `download` flag
|
||||||
|
|
||||||
|
`CaptureLibraryNodeRequest` gets `bool download = 3` (additive; old
|
||||||
|
clients keep bookmarking). `ProviderCommand::CaptureLibraryNode` carries
|
||||||
|
it and the handler picks the store. Error mapping extends the bookmark
|
||||||
|
contract: `InvalidName`/`BadSource` → `invalid_argument`,
|
||||||
|
`TooLarge`/`Disabled`/`Unsupported` → `failed_precondition`, download and
|
||||||
|
disk failures → `internal`.
|
||||||
|
|
||||||
|
### D6 — TUI: `W` on the library pane
|
||||||
|
|
||||||
|
`Action::LibraryDownloadNode` bound to `W` in `Scope::Library` ("Download
|
||||||
|
selection as capture"). Gate: the bare selection must be queueable **and**
|
||||||
|
downloadable (`selected_downloadable()`; child flag for nodes, the current
|
||||||
|
node's flag for tracks; marks ignored like `w`). The existing input
|
||||||
|
overlay opens with `InputPurpose::Capture { path, download: true }`
|
||||||
|
(label `capture`), prefilled with the selection title. Submit sends
|
||||||
|
`MessageFromUi::CaptureNode { path, name, download }` → the rpc. Failures
|
||||||
|
are logged, never fatal to the poll loop.
|
||||||
|
|
||||||
|
### D7 — Out of scope (explicitly)
|
||||||
|
|
||||||
|
- Retry/resume of failed or partial downloads (re-run the capture).
|
||||||
|
- Quality/codec selection, transcoding, tagging the audio files.
|
||||||
|
- Progress display in the TUI while a capture downloads.
|
||||||
|
- Deduplicating audio across captures, or refreshing links in existing
|
||||||
|
bookmarks into downloads.
|
||||||
|
- DRM circumvention: the download uses exactly the stream URLs the
|
||||||
|
provider already serves for playback.
|
||||||
|
|
||||||
|
## Structure
|
||||||
|
|
||||||
|
```d2
|
||||||
|
direction: right
|
||||||
|
|
||||||
|
server: crabidy-server {
|
||||||
|
orch: ProviderOrchestrator
|
||||||
|
cap: "capture.rs\nshared walk + caps + swap" {
|
||||||
|
link: "Sink::Link"
|
||||||
|
dl: "Sink::Download\n(reqwest, timeouts, byte budget)"
|
||||||
|
}
|
||||||
|
bs: BookmarkStore
|
||||||
|
cs: CaptureStore
|
||||||
|
}
|
||||||
|
|
||||||
|
tidal: "tidaldy\n(is_downloadable = is_queable)"
|
||||||
|
|
||||||
|
disk: "config/crabidy" {
|
||||||
|
shape: cylinder
|
||||||
|
b: "bookmarks/<name>/ (link tomls)"
|
||||||
|
c: "captures/<name>/ (audio + file tomls)"
|
||||||
|
}
|
||||||
|
|
||||||
|
cfs: "fsdy /captures\n(editable top level)"
|
||||||
|
|
||||||
|
server.orch -> server.bs: "CaptureLibraryNode\ndownload=false"
|
||||||
|
server.orch -> server.cs: "CaptureLibraryNode\ndownload=true"
|
||||||
|
server.bs -> server.cap.link
|
||||||
|
server.cs -> server.cap.dl
|
||||||
|
server.cap.dl -> tidal: "get_urls_for_track\n+ HTTP GET stream"
|
||||||
|
server.bs -> disk.b
|
||||||
|
server.cs -> disk.c
|
||||||
|
cfs -> disk.c: "list + parse (read only)"
|
||||||
|
server.orch -> cfs: "/captures/..."
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key flow: W on a Tidal album
|
||||||
|
|
||||||
|
```d2
|
||||||
|
shape: sequence_diagram
|
||||||
|
tui: TUI
|
||||||
|
rpc: gRPC
|
||||||
|
orch: Orchestrator
|
||||||
|
cs: CaptureStore
|
||||||
|
tidal: Tidal
|
||||||
|
|
||||||
|
tui -> rpc: "CaptureLibraryNode(path, name, download=true)"
|
||||||
|
rpc -> orch: "ProviderCommand (spawned)"
|
||||||
|
orch -> cs: "capture(name)"
|
||||||
|
cs -> orch: "get_lib_node: root allows download?"
|
||||||
|
cs -> tidal: "per track: get_urls_for_track"
|
||||||
|
cs -> tidal: "HTTP GET (deadline, byte budget)"
|
||||||
|
cs -> cs: "audio + toml pair\n(toml after audio, file = relative)"
|
||||||
|
cs -> rpc: "tmp-and-swap captures/<name>/"
|
||||||
|
rpc -> tui: OK
|
||||||
|
```
|
||||||
|
|
||||||
|
## Risks and open questions
|
||||||
|
|
||||||
|
- **Disk usage**: 500 tracks of FLAC can be tens of GiB; the byte budget
|
||||||
|
caps one capture, not the folder's total. Accepted — the user manages
|
||||||
|
`captures/` like any local music folder (and can delete via `d`).
|
||||||
|
- **Stream URL churn**: Tidal URLs are short-lived; the download happens
|
||||||
|
immediately after fetching each URL, so expiry only matters for very
|
||||||
|
slow transfers, which the per-track deadline already bounds.
|
||||||
|
- **Licensing**: captures are personal-use copies of streams the account
|
||||||
|
can already play; nothing here bypasses provider protection.
|
||||||
|
- Open (future): a progress event stream for long captures; retry with
|
||||||
|
classification + jitter; per-provider download quality knobs.
|
||||||
|
|
@ -72,6 +72,10 @@ pub enum Action {
|
||||||
/// capture the selected queueable subtree as a bookmark under
|
/// capture the selected queueable subtree as a bookmark under
|
||||||
/// `/bookmarks`. No-op unless the bare selection `is_queable`.
|
/// `/bookmarks`. No-op unless the bare selection `is_queable`.
|
||||||
LibraryCaptureNode,
|
LibraryCaptureNode,
|
||||||
|
/// Like [`Self::LibraryCaptureNode`], but the capture **downloads**
|
||||||
|
/// every track's audio into `/captures`. No-op unless the bare
|
||||||
|
/// selection is queueable *and* downloadable (e.g. Tidal subtrees).
|
||||||
|
LibraryDownloadNode,
|
||||||
// Queue pane
|
// Queue pane
|
||||||
QueueInsertHere,
|
QueueInsertHere,
|
||||||
QueueFirst,
|
QueueFirst,
|
||||||
|
|
@ -266,6 +270,13 @@ pub const BINDINGS: &[Binding] = &[
|
||||||
action: Action::LibraryCaptureNode,
|
action: Action::LibraryCaptureNode,
|
||||||
description: "Save selection as bookmark",
|
description: "Save selection as bookmark",
|
||||||
},
|
},
|
||||||
|
Binding {
|
||||||
|
scope: Scope::Library,
|
||||||
|
mods: KeyModifiers::SHIFT,
|
||||||
|
code: KeyCode::Char('W'),
|
||||||
|
action: Action::LibraryDownloadNode,
|
||||||
|
description: "Download selection as capture",
|
||||||
|
},
|
||||||
Binding {
|
Binding {
|
||||||
scope: Scope::Library,
|
scope: Scope::Library,
|
||||||
mods: KeyModifiers::NONE,
|
mods: KeyModifiers::NONE,
|
||||||
|
|
@ -563,6 +574,23 @@ mod tests {
|
||||||
),
|
),
|
||||||
Some(Action::LibraryCaptureNode)
|
Some(Action::LibraryCaptureNode)
|
||||||
);
|
);
|
||||||
|
// Shift-w is the download capture, library pane only.
|
||||||
|
assert_eq!(
|
||||||
|
lookup(
|
||||||
|
UiFocus::Library,
|
||||||
|
false,
|
||||||
|
key(KeyCode::Char('W'), KeyModifiers::SHIFT)
|
||||||
|
),
|
||||||
|
Some(Action::LibraryDownloadNode)
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
lookup(
|
||||||
|
UiFocus::Queue,
|
||||||
|
false,
|
||||||
|
key(KeyCode::Char('W'), KeyModifiers::SHIFT)
|
||||||
|
),
|
||||||
|
None
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
|
||||||
|
|
@ -74,6 +74,13 @@ impl Library {
|
||||||
item.is_queable
|
item.is_queable
|
||||||
.then(|| (item.path.clone(), item.title.clone()))
|
.then(|| (item.path.clone(), item.title.clone()))
|
||||||
}
|
}
|
||||||
|
/// The bare selection's path and title when it can be captured with a
|
||||||
|
/// download (`W`): queueable *and* downloadable. Marks are ignored,
|
||||||
|
/// like [`Self::selected_queueable`].
|
||||||
|
pub fn selected_downloadable(&self) -> Option<(String, String)> {
|
||||||
|
let item = self.list.get(self.list_state.selected()?)?;
|
||||||
|
(item.is_queable && item.is_downloadable).then(|| (item.path.clone(), item.title.clone()))
|
||||||
|
}
|
||||||
pub fn get_selected(&self) -> Option<Vec<String>> {
|
pub fn get_selected(&self) -> Option<Vec<String>> {
|
||||||
if self.list.iter().any(|i| i.marked) {
|
if self.list.iter().any(|i| i.marked) {
|
||||||
return Some(
|
return Some(
|
||||||
|
|
@ -188,6 +195,9 @@ impl Library {
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
is_editable: false,
|
is_editable: false,
|
||||||
is_deletable: false,
|
is_deletable: false,
|
||||||
|
// Tracks carry no wire flag: they inherit their node's
|
||||||
|
// blessing (architecture/captures.md D4).
|
||||||
|
is_downloadable: node.is_downloadable,
|
||||||
})
|
})
|
||||||
.chain(node.children.iter().map(|c| UiItem {
|
.chain(node.children.iter().map(|c| UiItem {
|
||||||
path: c.path.clone(),
|
path: c.path.clone(),
|
||||||
|
|
@ -198,6 +208,7 @@ impl Library {
|
||||||
is_creatable: c.is_creatable,
|
is_creatable: c.is_creatable,
|
||||||
is_editable: c.is_editable,
|
is_editable: c.is_editable,
|
||||||
is_deletable: c.is_deletable,
|
is_deletable: c.is_deletable,
|
||||||
|
is_downloadable: c.is_downloadable,
|
||||||
}))
|
}))
|
||||||
.collect();
|
.collect();
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -48,6 +48,9 @@ struct UiItem {
|
||||||
is_editable: bool,
|
is_editable: bool,
|
||||||
/// This item may be deleted (`d`) — part of the `[ed]` marker.
|
/// This item may be deleted (`d`) — part of the `[ed]` marker.
|
||||||
is_deletable: bool,
|
is_deletable: bool,
|
||||||
|
/// This item allows download captures (`W`). Tracks inherit their
|
||||||
|
/// containing node's flag; child nodes carry their own.
|
||||||
|
is_downloadable: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
pub const COLOR_PRIMARY: Color = Color::Rgb(129, 161, 193);
|
pub const COLOR_PRIMARY: Color = Color::Rgb(129, 161, 193);
|
||||||
|
|
@ -90,10 +93,13 @@ pub enum MessageFromUi {
|
||||||
/// as `/queues/<name>` in the library on the next visit).
|
/// as `/queues/<name>` in the library on the next visit).
|
||||||
SaveQueue(String),
|
SaveQueue(String),
|
||||||
/// Capture the queueable subtree at `path` as the bookmark `name`
|
/// Capture the queueable subtree at `path` as the bookmark `name`
|
||||||
/// (structure-preserving snapshot under `/bookmarks/<name>`).
|
/// (structure-preserving snapshot under `/bookmarks/<name>`), or —
|
||||||
|
/// with `download` — as the capture `name` under `/captures/<name>`
|
||||||
|
/// with every track's audio downloaded next to its toml.
|
||||||
CaptureNode {
|
CaptureNode {
|
||||||
path: String,
|
path: String,
|
||||||
name: String,
|
name: String,
|
||||||
|
download: bool,
|
||||||
},
|
},
|
||||||
AppendTracks(Vec<String>),
|
AppendTracks(Vec<String>),
|
||||||
QueueTracks(Vec<String>),
|
QueueTracks(Vec<String>),
|
||||||
|
|
@ -135,8 +141,10 @@ pub enum InputPurpose {
|
||||||
/// `w`: save the current queue under the entered name.
|
/// `w`: save the current queue under the entered name.
|
||||||
SaveQueue,
|
SaveQueue,
|
||||||
/// `w` in the library: capture the queueable subtree at `path` as a
|
/// `w` in the library: capture the queueable subtree at `path` as a
|
||||||
/// bookmark. The buffer starts prefilled with the selection's title.
|
/// bookmark — or, for `W` (`download`), as a download capture (the
|
||||||
Capture { path: String },
|
/// selection must also be downloadable). The buffer starts prefilled
|
||||||
|
/// with the selection's title.
|
||||||
|
Capture { path: String, download: bool },
|
||||||
}
|
}
|
||||||
|
|
||||||
/// State of the one-line text input overlay (node creation and rename).
|
/// State of the one-line text input overlay (node creation and rename).
|
||||||
|
|
@ -213,10 +221,11 @@ impl App {
|
||||||
InputPurpose::SaveQueue => {
|
InputPurpose::SaveQueue => {
|
||||||
let _ = self.tx.send(MessageFromUi::SaveQueue(title));
|
let _ = self.tx.send(MessageFromUi::SaveQueue(title));
|
||||||
}
|
}
|
||||||
InputPurpose::Capture { path } => {
|
InputPurpose::Capture { path, download } => {
|
||||||
let _ = self.tx.send(MessageFromUi::CaptureNode {
|
let _ = self.tx.send(MessageFromUi::CaptureNode {
|
||||||
path: path.clone(),
|
path: path.clone(),
|
||||||
name: title,
|
name: title,
|
||||||
|
download: *download,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -312,7 +321,23 @@ impl App {
|
||||||
// non-queueable selections like the other gated openers.
|
// non-queueable selections like the other gated openers.
|
||||||
if let Some((path, title)) = self.library.selected_queueable() {
|
if let Some((path, title)) = self.library.selected_queueable() {
|
||||||
self.input = Some(InputState {
|
self.input = Some(InputState {
|
||||||
purpose: InputPurpose::Capture { path },
|
purpose: InputPurpose::Capture {
|
||||||
|
path,
|
||||||
|
download: false,
|
||||||
|
},
|
||||||
|
buffer: title,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Action::LibraryDownloadNode => {
|
||||||
|
// `W`: like `w`, but the selection must also allow download
|
||||||
|
// captures (architecture/captures.md D6).
|
||||||
|
if let Some((path, title)) = self.library.selected_downloadable() {
|
||||||
|
self.input = Some(InputState {
|
||||||
|
purpose: InputPurpose::Capture {
|
||||||
|
path,
|
||||||
|
download: true,
|
||||||
|
},
|
||||||
buffer: title,
|
buffer: title,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
@ -389,7 +414,10 @@ impl App {
|
||||||
InputPurpose::Create { .. } => "new node",
|
InputPurpose::Create { .. } => "new node",
|
||||||
InputPurpose::Rename { .. } => "rename",
|
InputPurpose::Rename { .. } => "rename",
|
||||||
InputPurpose::SaveQueue => "save queue",
|
InputPurpose::SaveQueue => "save queue",
|
||||||
InputPurpose::Capture { .. } => "bookmark",
|
InputPurpose::Capture {
|
||||||
|
download: false, ..
|
||||||
|
} => "bookmark",
|
||||||
|
InputPurpose::Capture { download: true, .. } => "capture",
|
||||||
};
|
};
|
||||||
let line = Rect::new(area.x + 1, area.y + area.height - 2, area.width - 2, 1);
|
let line = Rect::new(area.x + 1, area.y + area.height - 2, area.width - 2, 1);
|
||||||
f.render_widget(Clear, line);
|
f.render_widget(Clear, line);
|
||||||
|
|
@ -482,6 +510,7 @@ mod tests {
|
||||||
tracks: Vec::new(),
|
tracks: Vec::new(),
|
||||||
is_queable: false,
|
is_queable: false,
|
||||||
is_creatable: true,
|
is_creatable: true,
|
||||||
|
is_downloadable: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -793,9 +822,13 @@ mod tests {
|
||||||
app.library.update(queueable_listing());
|
app.library.update(queueable_listing());
|
||||||
let _ = app.dispatch(Action::LibraryCaptureNode);
|
let _ = app.dispatch(Action::LibraryCaptureNode);
|
||||||
let input = app.input.as_ref().expect("capture overlay open");
|
let input = app.input.as_ref().expect("capture overlay open");
|
||||||
assert!(
|
assert!(matches!(
|
||||||
matches!(&input.purpose, InputPurpose::Capture { path } if path == "/tidal/artists/1")
|
&input.purpose,
|
||||||
);
|
InputPurpose::Capture {
|
||||||
|
path,
|
||||||
|
download: false
|
||||||
|
} if path == "/tidal/artists/1"
|
||||||
|
));
|
||||||
assert_eq!(input.buffer, "artist", "prefilled with the title");
|
assert_eq!(input.buffer, "artist", "prefilled with the title");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -809,9 +842,14 @@ mod tests {
|
||||||
app.handle_input_key(key(crossterm::event::KeyCode::Enter));
|
app.handle_input_key(key(crossterm::event::KeyCode::Enter));
|
||||||
assert!(app.input.is_none());
|
assert!(app.input.is_none());
|
||||||
match rx.try_recv() {
|
match rx.try_recv() {
|
||||||
Ok(MessageFromUi::CaptureNode { path, name }) => {
|
Ok(MessageFromUi::CaptureNode {
|
||||||
|
path,
|
||||||
|
name,
|
||||||
|
download,
|
||||||
|
}) => {
|
||||||
assert_eq!(path, "/tidal/artists/1");
|
assert_eq!(path, "/tidal/artists/1");
|
||||||
assert_eq!(name, "artist favs");
|
assert_eq!(name, "artist favs");
|
||||||
|
assert!(!download, "w captures a bookmark, not a download");
|
||||||
}
|
}
|
||||||
other => panic!("expected CaptureNode, got {:?}", other.is_ok()),
|
other => panic!("expected CaptureNode, got {:?}", other.is_ok()),
|
||||||
}
|
}
|
||||||
|
|
@ -822,6 +860,95 @@ mod tests {
|
||||||
assert!(rx.try_recv().is_err(), "Esc must not capture");
|
assert!(rx.try_recv().is_err(), "Esc must not capture");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Like [`queueable_listing`], but the child also allows download
|
||||||
|
/// captures (a Tidal subtree).
|
||||||
|
fn downloadable_listing() -> LibraryNode {
|
||||||
|
use crabidy_core::proto::crabidy::LibraryNodeChild;
|
||||||
|
LibraryNode {
|
||||||
|
children: vec![LibraryNodeChild {
|
||||||
|
is_downloadable: true,
|
||||||
|
..LibraryNodeChild::new("/tidal/artists/1".to_string(), "artist".to_string(), true)
|
||||||
|
}],
|
||||||
|
..creatable_node("/tidal/artists")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn download_capture_requires_the_downloadable_flag() {
|
||||||
|
let (mut app, _rx) = app();
|
||||||
|
// Queueable but not downloadable (e.g. an /fs folder): `W` must
|
||||||
|
// stay closed even though `w` would open.
|
||||||
|
app.library.update(queueable_listing());
|
||||||
|
let _ = app.dispatch(Action::LibraryDownloadNode);
|
||||||
|
assert!(app.input.is_none());
|
||||||
|
|
||||||
|
app.library.update(downloadable_listing());
|
||||||
|
let _ = app.dispatch(Action::LibraryDownloadNode);
|
||||||
|
let input = app.input.as_ref().expect("download overlay open");
|
||||||
|
assert!(matches!(
|
||||||
|
&input.purpose,
|
||||||
|
InputPurpose::Capture {
|
||||||
|
path,
|
||||||
|
download: true
|
||||||
|
} if path == "/tidal/artists/1"
|
||||||
|
));
|
||||||
|
assert_eq!(input.buffer, "artist", "prefilled with the title");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tracks_inherit_their_nodes_download_blessing() {
|
||||||
|
use crabidy_core::proto::crabidy::Track;
|
||||||
|
let track = Track {
|
||||||
|
path: "/tidal/artists/1/2/3".to_string(),
|
||||||
|
artist: "a".to_string(),
|
||||||
|
title: "one".to_string(),
|
||||||
|
duration: None,
|
||||||
|
album: None,
|
||||||
|
};
|
||||||
|
let listing = |downloadable| LibraryNode {
|
||||||
|
tracks: vec![track.clone()],
|
||||||
|
is_downloadable: downloadable,
|
||||||
|
..creatable_node("/tidal/artists/1/2")
|
||||||
|
};
|
||||||
|
let (mut app, _rx) = app();
|
||||||
|
app.library.update(listing(false));
|
||||||
|
let _ = app.dispatch(Action::LibraryDownloadNode);
|
||||||
|
assert!(app.input.is_none(), "unblessed node, unblessed tracks");
|
||||||
|
|
||||||
|
app.library.update(listing(true));
|
||||||
|
let _ = app.dispatch(Action::LibraryDownloadNode);
|
||||||
|
let input = app.input.as_ref().expect("download overlay open");
|
||||||
|
assert!(matches!(
|
||||||
|
&input.purpose,
|
||||||
|
InputPurpose::Capture {
|
||||||
|
path,
|
||||||
|
download: true
|
||||||
|
} if path == "/tidal/artists/1/2/3"
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn download_capture_submits_with_the_download_flag() {
|
||||||
|
let (mut app, rx) = app();
|
||||||
|
app.library.update(downloadable_listing());
|
||||||
|
let _ = app.dispatch(Action::LibraryDownloadNode);
|
||||||
|
type_str(&mut app, " local ");
|
||||||
|
app.handle_input_key(key(crossterm::event::KeyCode::Enter));
|
||||||
|
assert!(app.input.is_none());
|
||||||
|
match rx.try_recv() {
|
||||||
|
Ok(MessageFromUi::CaptureNode {
|
||||||
|
path,
|
||||||
|
name,
|
||||||
|
download,
|
||||||
|
}) => {
|
||||||
|
assert_eq!(path, "/tidal/artists/1");
|
||||||
|
assert_eq!(name, "artist local");
|
||||||
|
assert!(download, "W captures a download");
|
||||||
|
}
|
||||||
|
other => panic!("expected CaptureNode, got {:?}", other.is_ok()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// A one-track queue update, as the server would broadcast it.
|
/// A one-track queue update, as the server would broadcast it.
|
||||||
fn one_track_queue() -> crabidy_core::proto::crabidy::Queue {
|
fn one_track_queue() -> crabidy_core::proto::crabidy::Queue {
|
||||||
use crabidy_core::proto::crabidy::{Queue as ProtoQueue, Track};
|
use crabidy_core::proto::crabidy::{Queue as ProtoQueue, Track};
|
||||||
|
|
|
||||||
|
|
@ -81,6 +81,7 @@ impl Queue {
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
is_editable: false,
|
is_editable: false,
|
||||||
is_deletable: false,
|
is_deletable: false,
|
||||||
|
is_downloadable: false,
|
||||||
})
|
})
|
||||||
.collect();
|
.collect();
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -203,11 +203,14 @@ async fn poll(
|
||||||
error!(name, "failed to save queue: {err}");
|
error!(name, "failed to save queue: {err}");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
MessageFromUi::CaptureNode { path, name } => {
|
MessageFromUi::CaptureNode { path, name, download } => {
|
||||||
// A rejected capture (bad name, over-cap subtree) must
|
// A rejected capture (bad name, over-cap subtree, failed
|
||||||
// not tear down the poll loop either.
|
// download) must not tear down the poll loop either.
|
||||||
if let Err(err) = rpc_client.capture_library_node(path.clone(), name.clone()).await {
|
if let Err(err) = rpc_client
|
||||||
error!(path, name, "failed to capture subtree: {err}");
|
.capture_library_node(path.clone(), name.clone(), download)
|
||||||
|
.await
|
||||||
|
{
|
||||||
|
error!(path, name, download, "failed to capture subtree: {err}");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -225,8 +225,13 @@ impl RpcClient {
|
||||||
&mut self,
|
&mut self,
|
||||||
path: String,
|
path: String,
|
||||||
name: String,
|
name: String,
|
||||||
|
download: bool,
|
||||||
) -> Result<(), Box<dyn Error>> {
|
) -> Result<(), Box<dyn Error>> {
|
||||||
let capture_request = Request::new(CaptureLibraryNodeRequest { path, name });
|
let capture_request = Request::new(CaptureLibraryNodeRequest {
|
||||||
|
path,
|
||||||
|
name,
|
||||||
|
download,
|
||||||
|
});
|
||||||
self.client.capture_library_node(capture_request).await?;
|
self.client.capture_library_node(capture_request).await?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -105,8 +105,12 @@ message DeleteLibraryNodeResponse {
|
||||||
message CaptureLibraryNodeRequest {
|
message CaptureLibraryNodeRequest {
|
||||||
// Path of the queueable node (or track) to capture.
|
// Path of the queueable node (or track) to capture.
|
||||||
string path = 1;
|
string path = 1;
|
||||||
// Bookmark name; becomes the top-level folder under /bookmarks.
|
// Capture name; becomes the top-level folder under /bookmarks (or
|
||||||
|
// /captures when download is set).
|
||||||
string name = 2;
|
string name = 2;
|
||||||
|
// Download every track's audio into the capture instead of writing
|
||||||
|
// link files; the source node must set is_downloadable.
|
||||||
|
bool download = 3;
|
||||||
}
|
}
|
||||||
message CaptureLibraryNodeResponse {}
|
message CaptureLibraryNodeResponse {}
|
||||||
|
|
||||||
|
|
@ -207,6 +211,8 @@ message LibraryNodeChild {
|
||||||
bool is_editable = 5;
|
bool is_editable = 5;
|
||||||
// This node may be deleted (see DeleteLibraryNode).
|
// This node may be deleted (see DeleteLibraryNode).
|
||||||
bool is_deletable = 6;
|
bool is_deletable = 6;
|
||||||
|
// This node allows download captures (CaptureLibraryNode with download).
|
||||||
|
bool is_downloadable = 7;
|
||||||
}
|
}
|
||||||
|
|
||||||
message QueueModifiers {
|
message QueueModifiers {
|
||||||
|
|
@ -268,4 +274,7 @@ message LibraryNode {
|
||||||
bool is_queable = 6;
|
bool is_queable = 6;
|
||||||
// Children may be created under this node (see CreateLibraryNode).
|
// Children may be created under this node (see CreateLibraryNode).
|
||||||
bool is_creatable = 7;
|
bool is_creatable = 7;
|
||||||
|
// This node allows download captures; its listed tracks inherit the
|
||||||
|
// flag (CaptureLibraryNode with download).
|
||||||
|
bool is_downloadable = 8;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -209,14 +209,16 @@ impl LibraryNode {
|
||||||
tracks: Vec::new(),
|
tracks: Vec::new(),
|
||||||
is_queable: false,
|
is_queable: false,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
impl LibraryNodeChild {
|
impl LibraryNodeChild {
|
||||||
/// A regular, non-creatable, immutable child. Creatable/editable/
|
/// A regular, non-creatable, immutable child. Creatable/editable/
|
||||||
/// deletable children (e.g. the search node and its terms) set the
|
/// deletable/downloadable children (e.g. the search node and its
|
||||||
/// capability flags explicitly via struct update.
|
/// terms, or Tidal's queueable subtrees) set the capability flags
|
||||||
|
/// explicitly via struct update.
|
||||||
pub fn new(path: String, title: String, is_queable: bool) -> Self {
|
pub fn new(path: String, title: String, is_queable: bool) -> Self {
|
||||||
Self {
|
Self {
|
||||||
path,
|
path,
|
||||||
|
|
@ -225,6 +227,7 @@ impl LibraryNodeChild {
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
is_editable: false,
|
is_editable: false,
|
||||||
is_deletable: false,
|
is_deletable: false,
|
||||||
|
is_downloadable: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -361,6 +364,7 @@ mod tests {
|
||||||
.collect(),
|
.collect(),
|
||||||
is_queable,
|
is_queable,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -17,6 +17,7 @@ flume.workspace = true
|
||||||
fsdy.workspace = true
|
fsdy.workspace = true
|
||||||
futures.workspace = true
|
futures.workspace = true
|
||||||
rand.workspace = true
|
rand.workspace = true
|
||||||
|
reqwest.workspace = true
|
||||||
serde.workspace = true
|
serde.workspace = true
|
||||||
thiserror.workspace = true
|
thiserror.workspace = true
|
||||||
tidaldy.workspace = true
|
tidaldy.workspace = true
|
||||||
|
|
|
||||||
|
|
@ -17,10 +17,10 @@ pub const BOOKMARKS_PROVIDER_ROOT: &str = "/bookmarks";
|
||||||
|
|
||||||
/// The walk aborts beyond this many directories — a runaway provider tree
|
/// The walk aborts beyond this many directories — a runaway provider tree
|
||||||
/// must not fill the disk.
|
/// must not fill the disk.
|
||||||
pub const MAX_CAPTURE_DIRS: usize = 1_000;
|
pub const MAX_CAPTURE_DIRS: usize = crate::capture::BOOKMARK_CAPS.max_dirs;
|
||||||
|
|
||||||
/// The walk aborts beyond this many track files.
|
/// The walk aborts beyond this many track files.
|
||||||
pub const MAX_CAPTURE_TRACKS: usize = 20_000;
|
pub const MAX_CAPTURE_TRACKS: usize = crate::capture::BOOKMARK_CAPS.max_tracks;
|
||||||
|
|
||||||
/// The bookmarks directory: `bookmarks/` inside the crabidy config
|
/// The bookmarks directory: `bookmarks/` inside the crabidy config
|
||||||
/// directory. `None` when the platform has no config directory.
|
/// directory. `None` when the platform has no config directory.
|
||||||
|
|
@ -28,26 +28,7 @@ pub fn bookmarks_dir() -> Option<PathBuf> {
|
||||||
dirs::config_dir().map(|d| d.join("crabidy").join("bookmarks"))
|
dirs::config_dir().map(|d| d.join("crabidy").join("bookmarks"))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Errors from validating or writing a capture.
|
pub use crate::capture::CaptureError;
|
||||||
///
|
|
||||||
/// At the RPC boundary: `InvalidName`/`BadSource` → `invalid_argument`,
|
|
||||||
/// `TooLarge` → `failed_precondition`, the rest → `internal`. Messages
|
|
||||||
/// carry names and paths, never file contents.
|
|
||||||
#[derive(Debug, thiserror::Error)]
|
|
||||||
pub enum CaptureError {
|
|
||||||
#[error("invalid bookmark name: {0}")]
|
|
||||||
InvalidName(&'static str),
|
|
||||||
#[error("bookmarks are disabled")]
|
|
||||||
Disabled,
|
|
||||||
#[error("the source path cannot be captured: {0}")]
|
|
||||||
BadSource(String),
|
|
||||||
#[error("the subtree is too large to capture ({0})")]
|
|
||||||
TooLarge(&'static str),
|
|
||||||
#[error("cannot write bookmark: {0}")]
|
|
||||||
Io(#[from] std::io::Error),
|
|
||||||
#[error(transparent)]
|
|
||||||
TrackFile(#[from] fsdy::TrackFileError),
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Writes captured subtrees. All I/O is `tokio::fs`; the whole bookmark is
|
/// Writes captured subtrees. All I/O is `tokio::fs`; the whole bookmark is
|
||||||
/// built as a hidden temp sibling and swapped into place, so a crash never
|
/// built as a hidden temp sibling and swapped into place, so a crash never
|
||||||
|
|
@ -113,88 +94,20 @@ impl BookmarkStore {
|
||||||
where
|
where
|
||||||
C: ProviderClient + Sync,
|
C: ProviderClient + Sync,
|
||||||
{
|
{
|
||||||
let name = fsdy::validate_folder_name(name, &[]).map_err(CaptureError::InvalidName)?;
|
let caps = crate::capture::Caps {
|
||||||
let tmp = self.dir.join(format!(".tmp-{name}"));
|
max_dirs,
|
||||||
let written = self
|
max_tracks,
|
||||||
.write_capture(client, source_path, &tmp, max_dirs, max_tracks)
|
..crate::capture::BOOKMARK_CAPS
|
||||||
.await;
|
};
|
||||||
if let Err(err) = written {
|
crate::capture::capture_into(
|
||||||
// Every failure path removes the temp folder: nothing
|
&self.dir,
|
||||||
// half-written survives, not even hidden.
|
client,
|
||||||
let _ = tokio::fs::remove_dir_all(&tmp).await;
|
source_path,
|
||||||
return Err(err);
|
name,
|
||||||
}
|
caps,
|
||||||
let target = self.dir.join(name);
|
&crate::capture::Sink::Link,
|
||||||
if tokio::fs::try_exists(&target).await? {
|
)
|
||||||
tokio::fs::remove_dir_all(&target).await?;
|
|
||||||
}
|
|
||||||
tokio::fs::rename(&tmp, &target).await?;
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Builds the mirrored tree inside `tmp`. All-or-nothing: any provider
|
|
||||||
/// or write failure aborts the whole capture (the caller removes
|
|
||||||
/// `tmp`) — a bookmark that *looks* complete must *be* complete.
|
|
||||||
async fn write_capture<C>(
|
|
||||||
&self,
|
|
||||||
client: &C,
|
|
||||||
source_path: &str,
|
|
||||||
tmp: &Path,
|
|
||||||
max_dirs: usize,
|
|
||||||
max_tracks: usize,
|
|
||||||
) -> Result<(), CaptureError>
|
|
||||||
where
|
|
||||||
C: ProviderClient + Sync,
|
|
||||||
{
|
|
||||||
// A leftover temp folder from a crashed or racing capture is stale.
|
|
||||||
if tokio::fs::try_exists(tmp).await? {
|
|
||||||
tokio::fs::remove_dir_all(tmp).await?;
|
|
||||||
}
|
|
||||||
tokio::fs::create_dir_all(tmp).await?;
|
|
||||||
|
|
||||||
// A track source captures as a folder with one file.
|
|
||||||
if client.is_track_path(source_path) {
|
|
||||||
let track = client
|
|
||||||
.get_metadata_for_track(source_path)
|
|
||||||
.await
|
.await
|
||||||
.map_err(|err| CaptureError::BadSource(format!("{source_path}: {err}")))?;
|
|
||||||
let text = fsdy::TrackFile::from_track(&track).to_toml()?;
|
|
||||||
let file = tmp.join(fsdy::track_file_name(0, &track.title));
|
|
||||||
tokio::fs::write(file, text).await?;
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
|
|
||||||
let mut dirs = 1usize;
|
|
||||||
let mut tracks = 0usize;
|
|
||||||
// Iterative pre-order: a deep provider tree must not overflow the
|
|
||||||
// stack. Each entry pairs a library path with its mirror folder.
|
|
||||||
let mut worklist: Vec<(String, PathBuf)> =
|
|
||||||
vec![(source_path.to_string(), tmp.to_path_buf())];
|
|
||||||
while let Some((lib_path, dir)) = worklist.pop() {
|
|
||||||
let node = client
|
|
||||||
.get_lib_node(&lib_path)
|
|
||||||
.await
|
|
||||||
.map_err(|err| CaptureError::BadSource(format!("{lib_path}: {err}")))?;
|
|
||||||
for (index, track) in node.tracks.iter().enumerate() {
|
|
||||||
tracks += 1;
|
|
||||||
if tracks > max_tracks {
|
|
||||||
return Err(CaptureError::TooLarge("too many tracks"));
|
|
||||||
}
|
|
||||||
let text = fsdy::TrackFile::from_track(track).to_toml()?;
|
|
||||||
let file = dir.join(fsdy::track_file_name(index, &track.title));
|
|
||||||
tokio::fs::write(file, text).await?;
|
|
||||||
}
|
|
||||||
for (index, child) in node.children.iter().enumerate() {
|
|
||||||
dirs += 1;
|
|
||||||
if dirs > max_dirs {
|
|
||||||
return Err(CaptureError::TooLarge("too many directories"));
|
|
||||||
}
|
|
||||||
let child_dir = dir.join(fsdy::dir_name(index, &child.title));
|
|
||||||
tokio::fs::create_dir(&child_dir).await?;
|
|
||||||
worklist.push((child.path.clone(), child_dir));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,414 @@
|
||||||
|
//! The shared subtree-capture walk (see `architecture/captures.md` D2).
|
||||||
|
//!
|
||||||
|
//! Both bookmark captures (`w`, link files) and download captures (`W`,
|
||||||
|
//! audio files next to their tomls) mirror a library subtree into a folder:
|
||||||
|
//! an iterative pre-order walk over [`ProviderClient::get_lib_node`] with
|
||||||
|
//! order-prefixed names, size caps, a hidden tmp-and-swap write, and
|
||||||
|
//! all-or-nothing cleanup. This module owns that walk; the two stores only
|
||||||
|
//! differ in the per-track [`Sink`].
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
|
use crabidy_core::proto::crabidy::Track;
|
||||||
|
use crabidy_core::ProviderClient;
|
||||||
|
|
||||||
|
/// Connect timeout for download requests.
|
||||||
|
pub const DOWNLOAD_CONNECT_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
|
||||||
|
|
||||||
|
/// Total per-track deadline: URL fetch, request, and streaming the whole
|
||||||
|
/// body. A stalled transfer aborts the capture instead of hanging it.
|
||||||
|
pub const DOWNLOAD_TRACK_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(600);
|
||||||
|
|
||||||
|
/// Size limits for one capture. The walk aborts with
|
||||||
|
/// [`CaptureError::TooLarge`] when a limit trips — a runaway provider tree
|
||||||
|
/// or oversized stream must not fill the disk.
|
||||||
|
#[derive(Debug, Clone, Copy)]
|
||||||
|
pub struct Caps {
|
||||||
|
/// Maximum directories (the capture root counts as the first).
|
||||||
|
pub max_dirs: usize,
|
||||||
|
/// Maximum track files.
|
||||||
|
pub max_tracks: usize,
|
||||||
|
/// Maximum total downloaded bytes (ignored by [`Sink::Link`]).
|
||||||
|
pub max_bytes: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Caps for bookmark (link) captures: link files are tiny, so only the
|
||||||
|
/// tree size is bounded.
|
||||||
|
pub const BOOKMARK_CAPS: Caps = Caps {
|
||||||
|
max_dirs: 1_000,
|
||||||
|
max_tracks: 20_000,
|
||||||
|
max_bytes: u64::MAX,
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Caps for download captures: fewer tracks and a 4 GiB byte budget.
|
||||||
|
pub const DOWNLOAD_CAPS: Caps = Caps {
|
||||||
|
max_dirs: 1_000,
|
||||||
|
max_tracks: 500,
|
||||||
|
max_bytes: 4 * 1024 * 1024 * 1024,
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Errors from validating or writing a capture.
|
||||||
|
///
|
||||||
|
/// At the RPC boundary: `InvalidName`/`BadSource` → `invalid_argument`,
|
||||||
|
/// `TooLarge`/`Disabled`/`Unsupported` → `failed_precondition`, the rest →
|
||||||
|
/// `internal`. Messages carry names, paths, and counts, never file
|
||||||
|
/// contents.
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum CaptureError {
|
||||||
|
#[error("invalid name: {0}")]
|
||||||
|
InvalidName(&'static str),
|
||||||
|
#[error("the store is disabled")]
|
||||||
|
Disabled,
|
||||||
|
#[error("the source path cannot be captured: {0}")]
|
||||||
|
BadSource(String),
|
||||||
|
#[error("the source does not allow downloads")]
|
||||||
|
Unsupported,
|
||||||
|
#[error("the subtree is too large to capture ({0})")]
|
||||||
|
TooLarge(&'static str),
|
||||||
|
#[error("download failed: {0}")]
|
||||||
|
Download(String),
|
||||||
|
#[error("cannot write capture: {0}")]
|
||||||
|
Io(#[from] std::io::Error),
|
||||||
|
#[error(transparent)]
|
||||||
|
TrackFile(#[from] fsdy::TrackFileError),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What happens to each track the walk visits.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub enum Sink {
|
||||||
|
/// Write an order-prefixed link file ([`fsdy::TrackFile::from_track`])
|
||||||
|
/// — the bookmark behavior.
|
||||||
|
Link,
|
||||||
|
/// Download the track's audio next to an order-prefixed toml that
|
||||||
|
/// points at it — the captures behavior.
|
||||||
|
Download(Downloader),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Downloads one track's audio via the provider's stream URL.
|
||||||
|
///
|
||||||
|
/// One shared HTTP client with a connect timeout; each track is bounded by
|
||||||
|
/// [`DOWNLOAD_TRACK_TIMEOUT`] end to end and never retried (a capture is
|
||||||
|
/// re-runnable; overwrite = refresh). Bodies are streamed to disk against
|
||||||
|
/// the capture's remaining byte budget.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub struct Downloader {
|
||||||
|
http: reqwest::Client,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Downloader {
|
||||||
|
/// Builds the shared HTTP client. Fails only when the TLS backend
|
||||||
|
/// cannot initialize.
|
||||||
|
pub fn new() -> Result<Self, reqwest::Error> {
|
||||||
|
let http = reqwest::Client::builder()
|
||||||
|
.connect_timeout(DOWNLOAD_CONNECT_TIMEOUT)
|
||||||
|
.build()?;
|
||||||
|
Ok(Self { http })
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Downloads one track: audio file first, then the toml pointing at
|
||||||
|
/// it — a toml never exists without its audio. The whole operation is
|
||||||
|
/// bounded by [`DOWNLOAD_TRACK_TIMEOUT`].
|
||||||
|
///
|
||||||
|
/// Error messages carry the track's library path, never the stream
|
||||||
|
/// URL (it may embed a token) — reqwest errors are stripped with
|
||||||
|
/// [`reqwest::Error::without_url`].
|
||||||
|
async fn download_track<C>(
|
||||||
|
&self,
|
||||||
|
client: &C,
|
||||||
|
track: &Track,
|
||||||
|
dir: &Path,
|
||||||
|
index: usize,
|
||||||
|
bytes_left: &mut u64,
|
||||||
|
) -> Result<(), CaptureError>
|
||||||
|
where
|
||||||
|
C: ProviderClient + Sync,
|
||||||
|
{
|
||||||
|
let fetched = tokio::time::timeout(
|
||||||
|
DOWNLOAD_TRACK_TIMEOUT,
|
||||||
|
self.fetch_track(client, track, dir, index, bytes_left),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
match fetched {
|
||||||
|
Ok(result) => result,
|
||||||
|
Err(_) => Err(CaptureError::Download(format!(
|
||||||
|
"{}: timed out after {}s",
|
||||||
|
track.path,
|
||||||
|
DOWNLOAD_TRACK_TIMEOUT.as_secs()
|
||||||
|
))),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The unbounded body of [`Self::download_track`]: resolve the stream
|
||||||
|
/// URL, stream the response to disk against the byte budget, then
|
||||||
|
/// write the toml.
|
||||||
|
async fn fetch_track<C>(
|
||||||
|
&self,
|
||||||
|
client: &C,
|
||||||
|
track: &Track,
|
||||||
|
dir: &Path,
|
||||||
|
index: usize,
|
||||||
|
bytes_left: &mut u64,
|
||||||
|
) -> Result<(), CaptureError>
|
||||||
|
where
|
||||||
|
C: ProviderClient + Sync,
|
||||||
|
{
|
||||||
|
let track_path = track.path.as_str();
|
||||||
|
let urls = client
|
||||||
|
.get_urls_for_track(track_path)
|
||||||
|
.await
|
||||||
|
.map_err(|err| CaptureError::BadSource(format!("{track_path}: {err}")))?;
|
||||||
|
let url = urls
|
||||||
|
.first()
|
||||||
|
.ok_or_else(|| CaptureError::BadSource(format!("{track_path}: no stream url")))?;
|
||||||
|
let download_err = |err: reqwest::Error| {
|
||||||
|
CaptureError::Download(format!("{track_path}: {}", err.without_url()))
|
||||||
|
};
|
||||||
|
let mut response = self
|
||||||
|
.http
|
||||||
|
.get(url)
|
||||||
|
.send()
|
||||||
|
.await
|
||||||
|
.map_err(download_err)?
|
||||||
|
.error_for_status()
|
||||||
|
.map_err(download_err)?;
|
||||||
|
let content_type = response
|
||||||
|
.headers()
|
||||||
|
.get(reqwest::header::CONTENT_TYPE)
|
||||||
|
.and_then(|value| value.to_str().ok())
|
||||||
|
.map(str::to_string);
|
||||||
|
let ext = extension_for(content_type.as_deref(), url);
|
||||||
|
let audio_name = audio_file_name(index, &track.title, &ext);
|
||||||
|
let mut audio = tokio::fs::File::create(dir.join(&audio_name)).await?;
|
||||||
|
while let Some(chunk) = response.chunk().await.map_err(download_err)? {
|
||||||
|
let len = chunk.len() as u64;
|
||||||
|
if len > *bytes_left {
|
||||||
|
return Err(CaptureError::TooLarge("download budget exhausted"));
|
||||||
|
}
|
||||||
|
*bytes_left -= len;
|
||||||
|
tokio::io::AsyncWriteExt::write_all(&mut audio, &chunk).await?;
|
||||||
|
}
|
||||||
|
tokio::io::AsyncWriteExt::flush(&mut audio).await?;
|
||||||
|
drop(audio);
|
||||||
|
|
||||||
|
let text =
|
||||||
|
fsdy::TrackFile::from_track_with_file(track, Path::new(&audio_name)).to_toml()?;
|
||||||
|
tokio::fs::write(dir.join(fsdy::track_file_name(index, &track.title)), text).await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Captures the subtree at `source_path` as `dir/<name>/`, overwriting an
|
||||||
|
/// existing folder of that name.
|
||||||
|
///
|
||||||
|
/// Validates `name` ([`fsdy::validate_folder_name`], nothing reserved),
|
||||||
|
/// builds the whole capture in a hidden `.tmp-<name>` sibling via
|
||||||
|
/// [`write_tree`], and swaps it into place. Every failure removes the temp
|
||||||
|
/// folder — nothing half-written survives, not even hidden.
|
||||||
|
pub async fn capture_into<C>(
|
||||||
|
dir: &Path,
|
||||||
|
client: &C,
|
||||||
|
source_path: &str,
|
||||||
|
name: &str,
|
||||||
|
caps: Caps,
|
||||||
|
sink: &Sink,
|
||||||
|
) -> Result<(), CaptureError>
|
||||||
|
where
|
||||||
|
C: ProviderClient + Sync,
|
||||||
|
{
|
||||||
|
let name = fsdy::validate_folder_name(name, &[]).map_err(CaptureError::InvalidName)?;
|
||||||
|
let tmp = dir.join(format!(".tmp-{name}"));
|
||||||
|
let written = write_tree(client, source_path, &tmp, caps, sink).await;
|
||||||
|
if let Err(err) = written {
|
||||||
|
// Every failure path removes the temp folder: nothing
|
||||||
|
// half-written survives, not even hidden.
|
||||||
|
let _ = tokio::fs::remove_dir_all(&tmp).await;
|
||||||
|
return Err(err);
|
||||||
|
}
|
||||||
|
let target = dir.join(name);
|
||||||
|
if tokio::fs::try_exists(&target).await? {
|
||||||
|
tokio::fs::remove_dir_all(&target).await?;
|
||||||
|
}
|
||||||
|
tokio::fs::rename(&tmp, &target).await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Builds the mirrored tree inside `tmp`: iterative pre-order over
|
||||||
|
/// [`ProviderClient::get_lib_node`] (a deep tree must not overflow the
|
||||||
|
/// stack), every child node an order-prefixed folder ([`fsdy::dir_name`]),
|
||||||
|
/// every track handed to `sink` with its listing index. A `source_path`
|
||||||
|
/// that is a track captures as a folder with one entry. All-or-nothing:
|
||||||
|
/// any provider, download, or write failure aborts the whole capture.
|
||||||
|
async fn write_tree<C>(
|
||||||
|
client: &C,
|
||||||
|
source_path: &str,
|
||||||
|
tmp: &Path,
|
||||||
|
caps: Caps,
|
||||||
|
sink: &Sink,
|
||||||
|
) -> Result<(), CaptureError>
|
||||||
|
where
|
||||||
|
C: ProviderClient + Sync,
|
||||||
|
{
|
||||||
|
// A leftover temp folder from a crashed or racing capture is stale.
|
||||||
|
if tokio::fs::try_exists(tmp).await? {
|
||||||
|
tokio::fs::remove_dir_all(tmp).await?;
|
||||||
|
}
|
||||||
|
tokio::fs::create_dir_all(tmp).await?;
|
||||||
|
let mut bytes_left = caps.max_bytes;
|
||||||
|
|
||||||
|
// A track source captures as a folder with one entry.
|
||||||
|
if client.is_track_path(source_path) {
|
||||||
|
let track = client
|
||||||
|
.get_metadata_for_track(source_path)
|
||||||
|
.await
|
||||||
|
.map_err(|err| CaptureError::BadSource(format!("{source_path}: {err}")))?;
|
||||||
|
return write_track(sink, client, &track, tmp, 0, &mut bytes_left).await;
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut dirs = 1usize;
|
||||||
|
let mut tracks = 0usize;
|
||||||
|
// Iterative pre-order: a deep provider tree must not overflow the
|
||||||
|
// stack. Each entry pairs a library path with its mirror folder.
|
||||||
|
let mut worklist: Vec<(String, PathBuf)> = vec![(source_path.to_string(), tmp.to_path_buf())];
|
||||||
|
while let Some((lib_path, dir)) = worklist.pop() {
|
||||||
|
let node = client
|
||||||
|
.get_lib_node(&lib_path)
|
||||||
|
.await
|
||||||
|
.map_err(|err| CaptureError::BadSource(format!("{lib_path}: {err}")))?;
|
||||||
|
for (index, track) in node.tracks.iter().enumerate() {
|
||||||
|
tracks += 1;
|
||||||
|
if tracks > caps.max_tracks {
|
||||||
|
return Err(CaptureError::TooLarge("too many tracks"));
|
||||||
|
}
|
||||||
|
write_track(sink, client, track, &dir, index, &mut bytes_left).await?;
|
||||||
|
}
|
||||||
|
for (index, child) in node.children.iter().enumerate() {
|
||||||
|
dirs += 1;
|
||||||
|
if dirs > caps.max_dirs {
|
||||||
|
return Err(CaptureError::TooLarge("too many directories"));
|
||||||
|
}
|
||||||
|
let child_dir = dir.join(fsdy::dir_name(index, &child.title));
|
||||||
|
tokio::fs::create_dir(&child_dir).await?;
|
||||||
|
worklist.push((child.path.clone(), child_dir));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Writes one visited track into `dir` at listing position `index`,
|
||||||
|
/// drawing downloaded bytes from `bytes_left`.
|
||||||
|
async fn write_track<C>(
|
||||||
|
sink: &Sink,
|
||||||
|
client: &C,
|
||||||
|
track: &Track,
|
||||||
|
dir: &Path,
|
||||||
|
index: usize,
|
||||||
|
bytes_left: &mut u64,
|
||||||
|
) -> Result<(), CaptureError>
|
||||||
|
where
|
||||||
|
C: ProviderClient + Sync,
|
||||||
|
{
|
||||||
|
match sink {
|
||||||
|
Sink::Link => {
|
||||||
|
let text = fsdy::TrackFile::from_track(track).to_toml()?;
|
||||||
|
let file = dir.join(fsdy::track_file_name(index, &track.title));
|
||||||
|
tokio::fs::write(file, text).await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
Sink::Download(downloader) => {
|
||||||
|
downloader
|
||||||
|
.download_track(client, track, dir, index, bytes_left)
|
||||||
|
.await
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Picks the audio file extension: the response `Content-Type` first
|
||||||
|
/// (`audio/flac` → `flac`, `audio/mp4`/`audio/m4a` → `m4a`, `audio/mpeg` →
|
||||||
|
/// `mp3`, `audio/ogg` → `ogg`, `audio/wav` → `wav`), then the URL path's
|
||||||
|
/// extension, then `bin` — the player probes by content, the extension is
|
||||||
|
/// a hint.
|
||||||
|
fn extension_for(content_type: Option<&str>, url: &str) -> String {
|
||||||
|
let mapped = content_type
|
||||||
|
.and_then(|ct| ct.split(';').next())
|
||||||
|
.map(|essence| essence.trim().to_ascii_lowercase())
|
||||||
|
.and_then(|essence| match essence.as_str() {
|
||||||
|
"audio/flac" | "audio/x-flac" => Some("flac"),
|
||||||
|
"audio/mp4" | "audio/m4a" | "audio/x-m4a" => Some("m4a"),
|
||||||
|
"audio/mpeg" | "audio/mp3" => Some("mp3"),
|
||||||
|
"audio/ogg" => Some("ogg"),
|
||||||
|
"audio/wav" | "audio/x-wav" => Some("wav"),
|
||||||
|
_ => None,
|
||||||
|
});
|
||||||
|
match mapped.or_else(|| url_extension(url)) {
|
||||||
|
Some(ext) => ext.to_string(),
|
||||||
|
None => "bin".to_string(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The extension of a URL's last path segment (query and fragment
|
||||||
|
/// stripped), when it looks like one: short and alphanumeric.
|
||||||
|
fn url_extension(url: &str) -> Option<&str> {
|
||||||
|
let path = url.split(['?', '#']).next()?;
|
||||||
|
let segment = path.rsplit('/').next()?;
|
||||||
|
let (stem, ext) = segment.rsplit_once('.')?;
|
||||||
|
let plausible = !stem.is_empty()
|
||||||
|
&& !ext.is_empty()
|
||||||
|
&& ext.len() <= 5
|
||||||
|
&& ext.chars().all(|c| c.is_ascii_alphanumeric());
|
||||||
|
plausible.then_some(ext)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The audio file name for track `index` titled `title` with `ext`, next
|
||||||
|
/// to its toml: `NNNN <title>.<ext>` through the same sanitizer as the
|
||||||
|
/// toml name, so the pair sorts together — and what the toml's relative
|
||||||
|
/// `file` playable points at.
|
||||||
|
fn audio_file_name(index: usize, title: &str, ext: &str) -> String {
|
||||||
|
format!("{}.{ext}", fsdy::dir_name(index, title))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extension_prefers_content_type_then_url_then_bin() {
|
||||||
|
for (ct, ext) in [
|
||||||
|
("audio/flac", "flac"),
|
||||||
|
("audio/mp4", "m4a"),
|
||||||
|
("audio/m4a", "m4a"),
|
||||||
|
("audio/mpeg", "mp3"),
|
||||||
|
("audio/ogg", "ogg"),
|
||||||
|
("audio/wav", "wav"),
|
||||||
|
] {
|
||||||
|
assert_eq!(extension_for(Some(ct), "https://x.test/s"), ext);
|
||||||
|
}
|
||||||
|
// Content-Type parameters must not confuse the mapping.
|
||||||
|
assert_eq!(
|
||||||
|
extension_for(Some("audio/flac; charset=binary"), "https://x.test/s"),
|
||||||
|
"flac"
|
||||||
|
);
|
||||||
|
// Unknown or missing types fall back to the URL path's extension…
|
||||||
|
assert_eq!(
|
||||||
|
extension_for(None, "https://x.test/media/track.m4a?token=abc"),
|
||||||
|
"m4a"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
extension_for(Some("application/octet-stream"), "https://x.test/a.flac"),
|
||||||
|
"flac"
|
||||||
|
);
|
||||||
|
// …and to `bin` when the URL has none either.
|
||||||
|
assert_eq!(extension_for(None, "https://x.test/stream"), "bin");
|
||||||
|
assert_eq!(extension_for(None, "not a url"), "bin");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn audio_files_pair_with_their_toml_names() {
|
||||||
|
// Same prefix and sanitized stem as `fsdy::track_file_name`, so the
|
||||||
|
// audio file sorts right next to its toml.
|
||||||
|
assert_eq!(
|
||||||
|
audio_file_name(0, "We Will Rock You", "flac"),
|
||||||
|
"0001 We Will Rock You.flac"
|
||||||
|
);
|
||||||
|
assert_eq!(audio_file_name(11, "a/b", "mp3"), "0012 a_b.mp3");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,442 @@
|
||||||
|
//! Downloaded library subtrees ("captures") on disk
|
||||||
|
//! (see `architecture/captures.md`).
|
||||||
|
//!
|
||||||
|
//! Every capture is a folder under `<config>/crabidy/captures/` that
|
||||||
|
//! mirrors the captured subtree like a bookmark, except each track's audio
|
||||||
|
//! is **downloaded** next to its order-prefixed `*.cbd-track.toml`, and the
|
||||||
|
//! toml's playable is a relative `file` pointing at it — replaying a
|
||||||
|
//! capture needs no provider round trip. The same directory is mounted
|
||||||
|
//! read-only into the library as `/captures` by an `fsdy` instance (with
|
||||||
|
//! editable top-level folders); this module is the only writer.
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
|
use crabidy_core::ProviderClient;
|
||||||
|
|
||||||
|
use crate::capture::{Caps, CaptureError};
|
||||||
|
|
||||||
|
/// The library mount point of the captures directory.
|
||||||
|
pub const CAPTURES_PROVIDER_ROOT: &str = "/captures";
|
||||||
|
|
||||||
|
/// The captures directory: `captures/` inside the crabidy config
|
||||||
|
/// directory. `None` when the platform has no config directory.
|
||||||
|
pub fn captures_dir() -> Option<PathBuf> {
|
||||||
|
dirs::config_dir().map(|d| d.join("crabidy").join("captures"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Writes download captures. All audio is fetched through one shared HTTP
|
||||||
|
/// client; the whole capture is built as a hidden temp sibling and swapped
|
||||||
|
/// into place, so a crash or failed download never leaves a half-written
|
||||||
|
/// capture next to intact ones.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub struct CaptureStore {
|
||||||
|
dir: PathBuf,
|
||||||
|
sink: crate::capture::Sink,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl CaptureStore {
|
||||||
|
/// Opens the store at `dir`, creating the directory (and parents) if
|
||||||
|
/// missing, and builds the shared HTTP client.
|
||||||
|
pub async fn open(dir: PathBuf) -> Result<Self, CaptureError> {
|
||||||
|
tokio::fs::create_dir_all(&dir).await?;
|
||||||
|
let downloader = crate::capture::Downloader::new()
|
||||||
|
.map_err(|err| CaptureError::Download(format!("cannot build http client: {err}")))?;
|
||||||
|
Ok(Self {
|
||||||
|
dir,
|
||||||
|
sink: crate::capture::Sink::Download(downloader),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The store directory (what the `/captures` provider instance
|
||||||
|
/// mounts).
|
||||||
|
pub fn dir(&self) -> &Path {
|
||||||
|
&self.dir
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Captures the subtree at `source_path` as the capture `name`,
|
||||||
|
/// downloading every track's audio; overwrites an existing capture of
|
||||||
|
/// that name.
|
||||||
|
///
|
||||||
|
/// The capture **root** must opt in (`architecture/captures.md` D4): a
|
||||||
|
/// directory source must report `is_downloadable`, a track source's
|
||||||
|
/// parent node must — otherwise [`CaptureError::Unsupported`]. The
|
||||||
|
/// walk, caps ([`crate::capture::DOWNLOAD_CAPS`]), tmp-and-swap, and
|
||||||
|
/// all-or-nothing cleanup are [`crate::capture::capture_into`]'s.
|
||||||
|
pub async fn capture<C>(
|
||||||
|
&self,
|
||||||
|
client: &C,
|
||||||
|
source_path: &str,
|
||||||
|
name: &str,
|
||||||
|
) -> Result<(), CaptureError>
|
||||||
|
where
|
||||||
|
C: ProviderClient + Sync,
|
||||||
|
{
|
||||||
|
self.capture_with_caps(client, source_path, name, crate::capture::DOWNLOAD_CAPS)
|
||||||
|
.await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// [`Self::capture`] with explicit caps — the seam the cap tests use.
|
||||||
|
async fn capture_with_caps<C>(
|
||||||
|
&self,
|
||||||
|
client: &C,
|
||||||
|
source_path: &str,
|
||||||
|
name: &str,
|
||||||
|
caps: Caps,
|
||||||
|
) -> Result<(), CaptureError>
|
||||||
|
where
|
||||||
|
C: ProviderClient + Sync,
|
||||||
|
{
|
||||||
|
source_allows_download(client, source_path).await?;
|
||||||
|
crate::capture::capture_into(&self.dir, client, source_path, name, caps, &self.sink).await
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Checks that the capture root allows downloads: the node at
|
||||||
|
/// `source_path` (or, for a track path, its parent node) must set
|
||||||
|
/// `is_downloadable`. An unreadable root is [`CaptureError::BadSource`].
|
||||||
|
async fn source_allows_download<C>(client: &C, source_path: &str) -> Result<(), CaptureError>
|
||||||
|
where
|
||||||
|
C: ProviderClient + Sync,
|
||||||
|
{
|
||||||
|
let node_path = if client.is_track_path(source_path) {
|
||||||
|
crabidy_core::parent_path(source_path).unwrap_or(source_path)
|
||||||
|
} else {
|
||||||
|
source_path
|
||||||
|
};
|
||||||
|
let node = client
|
||||||
|
.get_lib_node(node_path)
|
||||||
|
.await
|
||||||
|
.map_err(|err| CaptureError::BadSource(format!("{node_path}: {err}")))?;
|
||||||
|
if node.is_downloadable {
|
||||||
|
Ok(())
|
||||||
|
} else {
|
||||||
|
Err(CaptureError::Unsupported)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::capture::{Caps, DOWNLOAD_CAPS};
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use crabidy_core::proto::crabidy::{LibraryNode, LibraryNodeChild, Track};
|
||||||
|
use crabidy_core::ProviderError;
|
||||||
|
use std::collections::HashMap;
|
||||||
|
use std::fs;
|
||||||
|
use std::path::Path;
|
||||||
|
use tempfile::TempDir;
|
||||||
|
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||||
|
|
||||||
|
/// Serves every request with one fixed response and returns a URL for
|
||||||
|
/// it. Minimal HTTP/1.1 on a loopback socket — enough for reqwest.
|
||||||
|
async fn serve(status: &'static str, content_type: &'static str, body: Vec<u8>) -> String {
|
||||||
|
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
|
||||||
|
.await
|
||||||
|
.expect("bind test server");
|
||||||
|
let addr = listener.local_addr().expect("test server addr");
|
||||||
|
tokio::spawn(async move {
|
||||||
|
while let Ok((mut sock, _)) = listener.accept().await {
|
||||||
|
let body = body.clone();
|
||||||
|
tokio::spawn(async move {
|
||||||
|
// Read until the header terminator; the request itself
|
||||||
|
// is irrelevant.
|
||||||
|
let mut buf = Vec::new();
|
||||||
|
let mut chunk = [0u8; 1024];
|
||||||
|
loop {
|
||||||
|
match sock.read(&mut chunk).await {
|
||||||
|
Ok(0) | Err(_) => break,
|
||||||
|
Ok(n) => {
|
||||||
|
buf.extend_from_slice(&chunk[..n]);
|
||||||
|
if buf.windows(4).any(|w| w == b"\r\n\r\n") {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let head = format!(
|
||||||
|
"HTTP/1.1 {status}\r\nContent-Type: {content_type}\r\n\
|
||||||
|
Content-Length: {}\r\nConnection: close\r\n\r\n",
|
||||||
|
body.len()
|
||||||
|
);
|
||||||
|
let _ = sock.write_all(head.as_bytes()).await;
|
||||||
|
let _ = sock.write_all(&body).await;
|
||||||
|
let _ = sock.shutdown().await;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
format!("http://{addr}/stream")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A provider with one album (`/mock/a`: tracks `one`, `two`) whose
|
||||||
|
/// stream URLs are the test server's, and a node-level download
|
||||||
|
/// blessing toggle.
|
||||||
|
#[derive(Debug)]
|
||||||
|
struct MockProvider {
|
||||||
|
urls: HashMap<String, String>,
|
||||||
|
downloadable: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl MockProvider {
|
||||||
|
fn new(urls: &[(&str, &str)], downloadable: bool) -> Self {
|
||||||
|
Self {
|
||||||
|
urls: urls
|
||||||
|
.iter()
|
||||||
|
.map(|(p, u)| (p.to_string(), u.to_string()))
|
||||||
|
.collect(),
|
||||||
|
downloadable,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn track(&self, path: &str) -> Track {
|
||||||
|
let title = match path {
|
||||||
|
"/mock/a/1" => "one",
|
||||||
|
"/mock/a/2" => "two",
|
||||||
|
other => other,
|
||||||
|
};
|
||||||
|
Track {
|
||||||
|
path: path.to_string(),
|
||||||
|
artist: "mock".to_string(),
|
||||||
|
title: title.to_string(),
|
||||||
|
duration: Some(10),
|
||||||
|
album: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl ProviderClient for MockProvider {
|
||||||
|
async fn init(_s: &str) -> Result<Self, ProviderError> {
|
||||||
|
Err(ProviderError::NotSupported)
|
||||||
|
}
|
||||||
|
fn settings(&self) -> String {
|
||||||
|
String::new()
|
||||||
|
}
|
||||||
|
fn is_track_path(&self, path: &str) -> bool {
|
||||||
|
path.starts_with("/mock/a/") && path.len() > "/mock/a/".len()
|
||||||
|
}
|
||||||
|
async fn get_urls_for_track(&self, track_path: &str) -> Result<Vec<String>, ProviderError> {
|
||||||
|
self.urls
|
||||||
|
.get(track_path)
|
||||||
|
.map(|u| vec![u.clone()])
|
||||||
|
.ok_or(ProviderError::FetchError)
|
||||||
|
}
|
||||||
|
async fn get_metadata_for_track(&self, track_path: &str) -> Result<Track, ProviderError> {
|
||||||
|
if !self.is_track_path(track_path) {
|
||||||
|
return Err(ProviderError::MalformedPath);
|
||||||
|
}
|
||||||
|
Ok(self.track(track_path))
|
||||||
|
}
|
||||||
|
fn get_lib_root(&self) -> LibraryNode {
|
||||||
|
LibraryNode::new()
|
||||||
|
}
|
||||||
|
async fn get_lib_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
|
||||||
|
let mut node = LibraryNode::new();
|
||||||
|
node.path = path.to_string();
|
||||||
|
node.is_downloadable = self.downloadable;
|
||||||
|
match path {
|
||||||
|
"/mock" => {
|
||||||
|
node.title = "mock".to_string();
|
||||||
|
node.children = vec![LibraryNodeChild {
|
||||||
|
is_downloadable: self.downloadable,
|
||||||
|
..LibraryNodeChild::new("/mock/a".to_string(), "a".to_string(), true)
|
||||||
|
}];
|
||||||
|
}
|
||||||
|
"/mock/a" => {
|
||||||
|
node.title = "a".to_string();
|
||||||
|
node.is_queable = true;
|
||||||
|
node.tracks = vec![self.track("/mock/a/1"), self.track("/mock/a/2")];
|
||||||
|
}
|
||||||
|
_ => return Err(ProviderError::MalformedPath),
|
||||||
|
}
|
||||||
|
Ok(node)
|
||||||
|
}
|
||||||
|
async fn create_lib_node(
|
||||||
|
&self,
|
||||||
|
_parent_path: &str,
|
||||||
|
_title: &str,
|
||||||
|
) -> Result<LibraryNode, ProviderError> {
|
||||||
|
Err(ProviderError::NotSupported)
|
||||||
|
}
|
||||||
|
async fn rename_lib_node(
|
||||||
|
&self,
|
||||||
|
_path: &str,
|
||||||
|
_new_title: &str,
|
||||||
|
) -> Result<LibraryNode, ProviderError> {
|
||||||
|
Err(ProviderError::NotSupported)
|
||||||
|
}
|
||||||
|
async fn delete_lib_node(&self, _path: &str) -> Result<LibraryNode, ProviderError> {
|
||||||
|
Err(ProviderError::NotSupported)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn store() -> (CaptureStore, TempDir) {
|
||||||
|
let dir = TempDir::new().expect("store tempdir");
|
||||||
|
let store = CaptureStore::open(dir.path().join("captures"))
|
||||||
|
.await
|
||||||
|
.expect("open creates the directory");
|
||||||
|
(store, dir)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn visible(dir: &Path) -> Vec<String> {
|
||||||
|
let mut names: Vec<String> = fs::read_dir(dir)
|
||||||
|
.expect("dir")
|
||||||
|
.map(|e| e.expect("entry").file_name().to_string_lossy().into_owned())
|
||||||
|
.filter(|n| !n.starts_with('.'))
|
||||||
|
.collect();
|
||||||
|
names.sort_by_key(|n| n.to_lowercase());
|
||||||
|
names
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn download_capture_writes_audio_next_to_pointing_tomls() {
|
||||||
|
let url = serve("200 OK", "audio/flac", b"flacbytes".to_vec()).await;
|
||||||
|
let mock = MockProvider::new(&[("/mock/a/1", &url), ("/mock/a/2", &url)], true);
|
||||||
|
let (store, _dir) = store().await;
|
||||||
|
store
|
||||||
|
.capture(&mock, "/mock/a", "faves")
|
||||||
|
.await
|
||||||
|
.expect("capture");
|
||||||
|
|
||||||
|
let root = store.dir().join("faves");
|
||||||
|
assert_eq!(
|
||||||
|
visible(&root),
|
||||||
|
vec![
|
||||||
|
"0001 one.cbd-track.toml".to_string(),
|
||||||
|
"0001 one.flac".into(),
|
||||||
|
"0002 two.cbd-track.toml".into(),
|
||||||
|
"0002 two.flac".into(),
|
||||||
|
]
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
fs::read(root.join("0001 one.flac")).expect("audio"),
|
||||||
|
b"flacbytes"
|
||||||
|
);
|
||||||
|
// The toml points at its sibling with a *relative* file playable.
|
||||||
|
let toml = fs::read_to_string(root.join("0002 two.cbd-track.toml")).expect("toml");
|
||||||
|
let file = fsdy::TrackFile::parse(&toml).expect("parses");
|
||||||
|
assert_eq!(
|
||||||
|
file.playable().expect("playable"),
|
||||||
|
fsdy::Playable::File("0002 two.flac".into())
|
||||||
|
);
|
||||||
|
assert_eq!(file.title, "two");
|
||||||
|
|
||||||
|
// Replay: the capture resolves through a /captures instance and the
|
||||||
|
// audio resolves to the absolute sibling path — no provider round
|
||||||
|
// trip left.
|
||||||
|
let captures = fsdy::Client::new(CAPTURES_PROVIDER_ROOT, store.dir().to_path_buf())
|
||||||
|
.expect("captures instance");
|
||||||
|
let (chunk_tx, chunk_rx) = flume::bounded(8);
|
||||||
|
captures
|
||||||
|
.resolve_tracks_into("/captures/faves", chunk_tx)
|
||||||
|
.await
|
||||||
|
.expect("resolve");
|
||||||
|
let tracks: Vec<Track> = chunk_rx.into_iter().flatten().collect();
|
||||||
|
assert_eq!(tracks.len(), 2);
|
||||||
|
let urls = captures
|
||||||
|
.get_urls_for_track(&tracks[0].path)
|
||||||
|
.await
|
||||||
|
.expect("urls");
|
||||||
|
assert_eq!(urls, vec![root.join("0001 one.flac").display().to_string()]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn capture_requires_the_root_download_blessing() {
|
||||||
|
let url = serve("200 OK", "audio/flac", b"x".to_vec()).await;
|
||||||
|
let mock = MockProvider::new(&[("/mock/a/1", &url), ("/mock/a/2", &url)], false);
|
||||||
|
let (store, _dir) = store().await;
|
||||||
|
assert!(matches!(
|
||||||
|
store.capture(&mock, "/mock/a", "faves").await,
|
||||||
|
Err(CaptureError::Unsupported)
|
||||||
|
));
|
||||||
|
assert!(matches!(
|
||||||
|
store.capture(&mock, "/mock/a/1", "faves").await,
|
||||||
|
Err(CaptureError::Unsupported)
|
||||||
|
));
|
||||||
|
assert_eq!(fs::read_dir(store.dir()).expect("store dir").count(), 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn capturing_a_single_blessed_track_writes_one_pair() {
|
||||||
|
let url = serve("200 OK", "audio/mpeg", b"mp3bytes".to_vec()).await;
|
||||||
|
let mock = MockProvider::new(&[("/mock/a/1", &url)], true);
|
||||||
|
let (store, _dir) = store().await;
|
||||||
|
store
|
||||||
|
.capture(&mock, "/mock/a/1", "just one")
|
||||||
|
.await
|
||||||
|
.expect("capture track");
|
||||||
|
assert_eq!(
|
||||||
|
visible(&store.dir().join("just one")),
|
||||||
|
vec!["0001 one.cbd-track.toml".to_string(), "0001 one.mp3".into()]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn download_capture_is_all_or_nothing() {
|
||||||
|
let ok = serve("200 OK", "audio/flac", b"x".to_vec()).await;
|
||||||
|
let gone = serve("404 Not Found", "text/plain", Vec::new()).await;
|
||||||
|
let mock = MockProvider::new(&[("/mock/a/1", &ok), ("/mock/a/2", &gone)], true);
|
||||||
|
let (store, _dir) = store().await;
|
||||||
|
let err = store
|
||||||
|
.capture(&mock, "/mock/a", "faves")
|
||||||
|
.await
|
||||||
|
.expect_err("a failed download aborts the capture");
|
||||||
|
assert!(matches!(err, CaptureError::Download(_)), "got {err:?}");
|
||||||
|
// Nothing half-written survives, not even hidden temp folders.
|
||||||
|
assert_eq!(fs::read_dir(store.dir()).expect("store dir").count(), 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn download_capture_enforces_its_caps() {
|
||||||
|
let url = serve("200 OK", "audio/flac", b"0123456789".to_vec()).await;
|
||||||
|
let mock = MockProvider::new(&[("/mock/a/1", &url), ("/mock/a/2", &url)], true);
|
||||||
|
let (store, _dir) = store().await;
|
||||||
|
// Byte budget: two 10-byte bodies against a 15-byte budget.
|
||||||
|
let caps = Caps {
|
||||||
|
max_bytes: 15,
|
||||||
|
..DOWNLOAD_CAPS
|
||||||
|
};
|
||||||
|
let err = store
|
||||||
|
.capture_with_caps(&mock, "/mock/a", "big", caps)
|
||||||
|
.await
|
||||||
|
.expect_err("over the byte budget");
|
||||||
|
assert!(matches!(err, CaptureError::TooLarge(_)), "got {err:?}");
|
||||||
|
// Track cap.
|
||||||
|
let caps = Caps {
|
||||||
|
max_tracks: 1,
|
||||||
|
..DOWNLOAD_CAPS
|
||||||
|
};
|
||||||
|
let err = store
|
||||||
|
.capture_with_caps(&mock, "/mock/a", "big", caps)
|
||||||
|
.await
|
||||||
|
.expect_err("over the track cap");
|
||||||
|
assert!(matches!(err, CaptureError::TooLarge(_)), "got {err:?}");
|
||||||
|
assert_eq!(fs::read_dir(store.dir()).expect("store dir").count(), 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn capture_validates_names_and_overwrites() {
|
||||||
|
let url = serve("200 OK", "audio/flac", b"x".to_vec()).await;
|
||||||
|
let mock = MockProvider::new(&[("/mock/a/1", &url), ("/mock/a/2", &url)], true);
|
||||||
|
let (store, _dir) = store().await;
|
||||||
|
assert!(matches!(
|
||||||
|
store.capture(&mock, "/mock/a", "a/b").await,
|
||||||
|
Err(CaptureError::InvalidName(_))
|
||||||
|
));
|
||||||
|
store
|
||||||
|
.capture(&mock, "/mock/a", "faves")
|
||||||
|
.await
|
||||||
|
.expect("first capture");
|
||||||
|
store
|
||||||
|
.capture(&mock, "/mock/a/1", "faves")
|
||||||
|
.await
|
||||||
|
.expect("overwrite");
|
||||||
|
// The overwrite fully replaces the older, larger capture.
|
||||||
|
assert_eq!(
|
||||||
|
visible(&store.dir().join("faves")),
|
||||||
|
vec![
|
||||||
|
"0001 one.cbd-track.toml".to_string(),
|
||||||
|
"0001 one.flac".into()
|
||||||
|
]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -1,4 +1,6 @@
|
||||||
pub mod bookmark_store;
|
pub mod bookmark_store;
|
||||||
|
pub mod capture;
|
||||||
|
pub mod capture_store;
|
||||||
pub mod queue_store;
|
pub mod queue_store;
|
||||||
|
|
||||||
use crabidy_core::proto::crabidy::{Queue, Track};
|
use crabidy_core::proto::crabidy::{Queue, Track};
|
||||||
|
|
|
||||||
|
|
@ -205,12 +205,16 @@ pub enum ProviderCommand {
|
||||||
result_tx: flume::Sender<Result<LibraryNode, ProviderError>>,
|
result_tx: flume::Sender<Result<LibraryNode, ProviderError>>,
|
||||||
},
|
},
|
||||||
/// Captures the queueable subtree at `path` as the bookmark `name`
|
/// Captures the queueable subtree at `path` as the bookmark `name`
|
||||||
/// (see `architecture/bookmarks.md` D1–D3). Handled on a spawned task —
|
/// (see `architecture/bookmarks.md` D1–D3) — or, with `download`, as
|
||||||
/// a large walk must not block the orchestrator loop.
|
/// the download capture `name` under `/captures`, fetching every
|
||||||
|
/// track's audio (see `architecture/captures.md`). Handled on a
|
||||||
|
/// spawned task — a large walk or download must not block the
|
||||||
|
/// orchestrator loop.
|
||||||
CaptureLibraryNode {
|
CaptureLibraryNode {
|
||||||
path: String,
|
path: String,
|
||||||
name: String,
|
name: String,
|
||||||
result_tx: flume::Sender<Result<(), crabidy_server::bookmark_store::CaptureError>>,
|
download: bool,
|
||||||
|
result_tx: flume::Sender<Result<(), crabidy_server::capture::CaptureError>>,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,7 @@ use crabidy_core::{
|
||||||
ProviderClient, ProviderError,
|
ProviderClient, ProviderError,
|
||||||
};
|
};
|
||||||
use crabidy_server::bookmark_store::{BookmarkStore, BOOKMARKS_PROVIDER_ROOT};
|
use crabidy_server::bookmark_store::{BookmarkStore, BOOKMARKS_PROVIDER_ROOT};
|
||||||
|
use crabidy_server::capture_store::{CaptureStore, CAPTURES_PROVIDER_ROOT};
|
||||||
use crabidy_server::queue_store::{CURRENT_QUEUE_NAME, QUEUES_PROVIDER_ROOT};
|
use crabidy_server::queue_store::{CURRENT_QUEUE_NAME, QUEUES_PROVIDER_ROOT};
|
||||||
use std::{fs, path::PathBuf, sync::Arc};
|
use std::{fs, path::PathBuf, sync::Arc};
|
||||||
use tracing::{debug, debug_span, error, instrument, warn, Instrument};
|
use tracing::{debug, debug_span, error, instrument, warn, Instrument};
|
||||||
|
|
@ -25,9 +26,16 @@ pub struct ProviderOrchestrator {
|
||||||
/// `/bookmarks` (architecture/bookmarks.md D1). `None` without a
|
/// `/bookmarks` (architecture/bookmarks.md D1). `None` without a
|
||||||
/// config directory.
|
/// config directory.
|
||||||
bookmarks_client: Option<Arc<fsdy::Client>>,
|
bookmarks_client: Option<Arc<fsdy::Client>>,
|
||||||
/// The capture writer; `None` disables `CaptureLibraryNode` (and the
|
/// The bookmark writer; `None` disables link captures (and the
|
||||||
/// `/bookmarks` mount goes with it).
|
/// `/bookmarks` mount goes with it).
|
||||||
bookmark_store: Option<Arc<BookmarkStore>>,
|
bookmark_store: Option<Arc<BookmarkStore>>,
|
||||||
|
/// Fourth `fsdy` instance over the captures folder, mounted at
|
||||||
|
/// `/captures` (architecture/captures.md D1). `None` without a config
|
||||||
|
/// directory.
|
||||||
|
captures_client: Option<Arc<fsdy::Client>>,
|
||||||
|
/// The download-capture writer; `None` disables download captures
|
||||||
|
/// (and the `/captures` mount goes with it).
|
||||||
|
capture_store: Option<Arc<CaptureStore>>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether a path belongs to the filesystem provider.
|
/// Whether a path belongs to the filesystem provider.
|
||||||
|
|
@ -46,6 +54,11 @@ fn bookmarks_owns(path: &str) -> bool {
|
||||||
path == BOOKMARKS_PROVIDER_ROOT || path.starts_with("/bookmarks/")
|
path == BOOKMARKS_PROVIDER_ROOT || path.starts_with("/bookmarks/")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Whether a path belongs to the captures provider instance.
|
||||||
|
fn captures_owns(path: &str) -> bool {
|
||||||
|
path == CAPTURES_PROVIDER_ROOT || path.starts_with("/captures/")
|
||||||
|
}
|
||||||
|
|
||||||
impl ProviderOrchestrator {
|
impl ProviderOrchestrator {
|
||||||
/// The fs client, or `MalformedPath` (with a warning) when the
|
/// The fs client, or `MalformedPath` (with a warning) when the
|
||||||
/// provider is disabled — a `/fs` path then has no owner.
|
/// provider is disabled — a `/fs` path then has no owner.
|
||||||
|
|
@ -73,6 +86,15 @@ impl ProviderOrchestrator {
|
||||||
ProviderError::MalformedPath
|
ProviderError::MalformedPath
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The captures client, or `MalformedPath` (with a warning) when the
|
||||||
|
/// instance is disabled — a `/captures` path then has no owner.
|
||||||
|
fn captures_provider(&self) -> Result<&fsdy::Client, ProviderError> {
|
||||||
|
self.captures_client.as_deref().ok_or_else(|| {
|
||||||
|
warn!("captures library is disabled");
|
||||||
|
ProviderError::MalformedPath
|
||||||
|
})
|
||||||
|
}
|
||||||
pub fn run(self) {
|
pub fn run(self) {
|
||||||
tokio::spawn(async move {
|
tokio::spawn(async move {
|
||||||
// Behind an Arc so long-running resolves can be spawned onto
|
// Behind an Arc so long-running resolves can be spawned onto
|
||||||
|
|
@ -148,20 +170,29 @@ impl ProviderOrchestrator {
|
||||||
ProviderCommand::CaptureLibraryNode {
|
ProviderCommand::CaptureLibraryNode {
|
||||||
path,
|
path,
|
||||||
name,
|
name,
|
||||||
|
download,
|
||||||
result_tx,
|
result_tx,
|
||||||
} => {
|
} => {
|
||||||
// Spawned: capturing a large artist walks many provider
|
// Spawned: capturing a large artist walks many provider
|
||||||
// nodes and must not block this loop (the walk itself
|
// nodes (and, for downloads, streams audio) and must not
|
||||||
// calls back into `get_lib_node` via `this`).
|
// block this loop (the walk itself calls back into
|
||||||
|
// `get_lib_node` via `this`).
|
||||||
let this = Arc::clone(&self);
|
let this = Arc::clone(&self);
|
||||||
tokio::spawn(
|
tokio::spawn(
|
||||||
async move {
|
async move {
|
||||||
let result = match &this.bookmark_store {
|
let result = if download {
|
||||||
|
match &this.capture_store {
|
||||||
Some(store) => store.capture(&*this, &path, &name).await,
|
Some(store) => store.capture(&*this, &path, &name).await,
|
||||||
None => Err(crabidy_server::bookmark_store::CaptureError::Disabled),
|
None => Err(crabidy_server::capture::CaptureError::Disabled),
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
match &this.bookmark_store {
|
||||||
|
Some(store) => store.capture(&*this, &path, &name).await,
|
||||||
|
None => Err(crabidy_server::capture::CaptureError::Disabled),
|
||||||
|
}
|
||||||
};
|
};
|
||||||
if let Err(err) = &result {
|
if let Err(err) = &result {
|
||||||
warn!(path, name, "cannot capture subtree: {err}");
|
warn!(path, name, download, "cannot capture subtree: {err}");
|
||||||
}
|
}
|
||||||
if let Err(err) = result_tx.send_async(result).await {
|
if let Err(err) = result_tx.send_async(result).await {
|
||||||
error!("failed to send capture_library_node result: {err}");
|
error!("failed to send capture_library_node result: {err}");
|
||||||
|
|
@ -263,6 +294,30 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
// Captures: like bookmarks, but the store downloads every track's
|
||||||
|
// audio next to its toml (architecture/captures.md D1). Non-fatal.
|
||||||
|
let capture_store = match crabidy_server::capture_store::captures_dir() {
|
||||||
|
Some(dir) => match CaptureStore::open(dir).await {
|
||||||
|
Ok(store) => Some(Arc::new(store)),
|
||||||
|
Err(err) => {
|
||||||
|
warn!("captures disabled: {err}");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
},
|
||||||
|
None => {
|
||||||
|
warn!("captures disabled: no config directory");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let captures_client = capture_store.as_ref().and_then(|store| {
|
||||||
|
match fsdy::Client::new(CAPTURES_PROVIDER_ROOT, store.dir().to_path_buf()) {
|
||||||
|
Ok(client) => Some(Arc::new(client.with_editable_top_level(&[]))),
|
||||||
|
Err(err) => {
|
||||||
|
warn!("captures library disabled: {err}");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
let (provider_tx, provider_rx) = flume::bounded(100);
|
let (provider_tx, provider_rx) = flume::bounded(100);
|
||||||
Ok(Self {
|
Ok(Self {
|
||||||
provider_rx,
|
provider_rx,
|
||||||
|
|
@ -272,6 +327,8 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
queues_client,
|
queues_client,
|
||||||
bookmarks_client,
|
bookmarks_client,
|
||||||
bookmark_store,
|
bookmark_store,
|
||||||
|
captures_client,
|
||||||
|
capture_store,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -302,6 +359,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.as_ref()
|
.as_ref()
|
||||||
.is_some_and(|bookmarks| bookmarks.is_track_path(path));
|
.is_some_and(|bookmarks| bookmarks.is_track_path(path));
|
||||||
}
|
}
|
||||||
|
if captures_owns(path) {
|
||||||
|
return self
|
||||||
|
.captures_client
|
||||||
|
.as_ref()
|
||||||
|
.is_some_and(|captures| captures.is_track_path(path));
|
||||||
|
}
|
||||||
false
|
false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -322,6 +385,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.get_urls_for_track(track_path)
|
.get_urls_for_track(track_path)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
if captures_owns(track_path) {
|
||||||
|
return self
|
||||||
|
.captures_provider()?
|
||||||
|
.get_urls_for_track(track_path)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
warn!(path = track_path, "no provider owns this track path");
|
warn!(path = track_path, "no provider owns this track path");
|
||||||
Err(ProviderError::MalformedPath)
|
Err(ProviderError::MalformedPath)
|
||||||
}
|
}
|
||||||
|
|
@ -346,6 +415,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.get_metadata_for_track(track_path)
|
.get_metadata_for_track(track_path)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
if captures_owns(track_path) {
|
||||||
|
return self
|
||||||
|
.captures_provider()?
|
||||||
|
.get_metadata_for_track(track_path)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
warn!(path = track_path, "no provider owns this track path");
|
warn!(path = track_path, "no provider owns this track path");
|
||||||
Err(ProviderError::MalformedPath)
|
Err(ProviderError::MalformedPath)
|
||||||
}
|
}
|
||||||
|
|
@ -373,6 +448,14 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
);
|
);
|
||||||
root_node.children.push(child);
|
root_node.children.push(child);
|
||||||
}
|
}
|
||||||
|
if self.captures_client.is_some() {
|
||||||
|
let child = LibraryNodeChild::new(
|
||||||
|
CAPTURES_PROVIDER_ROOT.to_owned(),
|
||||||
|
"captures".to_owned(),
|
||||||
|
false,
|
||||||
|
);
|
||||||
|
root_node.children.push(child);
|
||||||
|
}
|
||||||
root_node
|
root_node
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -394,6 +477,9 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
if bookmarks_owns(path) {
|
if bookmarks_owns(path) {
|
||||||
return self.bookmarks_provider()?.get_lib_node(path).await;
|
return self.bookmarks_provider()?.get_lib_node(path).await;
|
||||||
}
|
}
|
||||||
|
if captures_owns(path) {
|
||||||
|
return self.captures_provider()?.get_lib_node(path).await;
|
||||||
|
}
|
||||||
warn!(path, "no provider owns this path");
|
warn!(path, "no provider owns this path");
|
||||||
Err(ProviderError::MalformedPath)
|
Err(ProviderError::MalformedPath)
|
||||||
}
|
}
|
||||||
|
|
@ -427,6 +513,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.create_lib_node(parent_path, title)
|
.create_lib_node(parent_path, title)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
if captures_owns(parent_path) {
|
||||||
|
return self
|
||||||
|
.captures_provider()?
|
||||||
|
.create_lib_node(parent_path, title)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
warn!(parent_path, "no provider supports creating nodes here");
|
warn!(parent_path, "no provider supports creating nodes here");
|
||||||
Err(ProviderError::NotSupported)
|
Err(ProviderError::NotSupported)
|
||||||
}
|
}
|
||||||
|
|
@ -457,6 +549,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.rename_lib_node(path, new_title)
|
.rename_lib_node(path, new_title)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
if captures_owns(path) {
|
||||||
|
return self
|
||||||
|
.captures_provider()?
|
||||||
|
.rename_lib_node(path, new_title)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
warn!(path, "no provider supports renaming this node");
|
warn!(path, "no provider supports renaming this node");
|
||||||
Err(ProviderError::NotSupported)
|
Err(ProviderError::NotSupported)
|
||||||
}
|
}
|
||||||
|
|
@ -490,6 +588,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.resolve_tracks_into(path, chunk_tx)
|
.resolve_tracks_into(path, chunk_tx)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
if captures_owns(path) {
|
||||||
|
return self
|
||||||
|
.captures_provider()?
|
||||||
|
.resolve_tracks_into(path, chunk_tx)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
warn!(path, "no provider owns this path");
|
warn!(path, "no provider owns this path");
|
||||||
Err(ProviderError::MalformedPath)
|
Err(ProviderError::MalformedPath)
|
||||||
}
|
}
|
||||||
|
|
@ -510,6 +614,9 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
if bookmarks_owns(path) {
|
if bookmarks_owns(path) {
|
||||||
return self.bookmarks_provider()?.delete_lib_node(path).await;
|
return self.bookmarks_provider()?.delete_lib_node(path).await;
|
||||||
}
|
}
|
||||||
|
if captures_owns(path) {
|
||||||
|
return self.captures_provider()?.delete_lib_node(path).await;
|
||||||
|
}
|
||||||
warn!(path, "no provider supports deleting this node");
|
warn!(path, "no provider supports deleting this node");
|
||||||
Err(ProviderError::NotSupported)
|
Err(ProviderError::NotSupported)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -377,26 +377,33 @@ impl CrabidyService for RpcService {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Captures a queueable subtree as a bookmark via the provider loop
|
/// Captures a queueable subtree as a bookmark via the provider loop
|
||||||
/// (structure-preserving snapshot under `/bookmarks/<name>`).
|
/// (structure-preserving snapshot under `/bookmarks/<name>`), or —
|
||||||
|
/// with `download` — as a download capture under `/captures/<name>`.
|
||||||
///
|
///
|
||||||
/// Error mapping is part of the contract: invalid name or
|
/// Error mapping is part of the contract: invalid name or
|
||||||
/// uncapturable source → `invalid_argument`; an over-cap subtree or
|
/// uncapturable source → `invalid_argument`; an over-cap subtree,
|
||||||
/// disabled bookmarks → `failed_precondition`; walk/write failures →
|
/// disabled store, or a source that does not allow downloads →
|
||||||
/// `internal`.
|
/// `failed_precondition`; walk/write/download failures → `internal`.
|
||||||
#[instrument(skip(self, request), fields(path, name))]
|
#[instrument(skip(self, request), fields(path, name, download))]
|
||||||
async fn capture_library_node(
|
async fn capture_library_node(
|
||||||
&self,
|
&self,
|
||||||
request: Request<CaptureLibraryNodeRequest>,
|
request: Request<CaptureLibraryNodeRequest>,
|
||||||
) -> Result<Response<CaptureLibraryNodeResponse>, Status> {
|
) -> Result<Response<CaptureLibraryNodeResponse>, Status> {
|
||||||
let CaptureLibraryNodeRequest { path, name } = request.into_inner();
|
let CaptureLibraryNodeRequest {
|
||||||
|
path,
|
||||||
|
name,
|
||||||
|
download,
|
||||||
|
} = request.into_inner();
|
||||||
tracing::Span::current().record("path", path.as_str());
|
tracing::Span::current().record("path", path.as_str());
|
||||||
tracing::Span::current().record("name", name.as_str());
|
tracing::Span::current().record("name", name.as_str());
|
||||||
|
tracing::Span::current().record("download", download);
|
||||||
debug!("received capture_library_node request");
|
debug!("received capture_library_node request");
|
||||||
let (result_tx, result_rx) = flume::bounded(1);
|
let (result_tx, result_rx) = flume::bounded(1);
|
||||||
self.provider_tx
|
self.provider_tx
|
||||||
.send_async(ProviderMessage::new(ProviderCommand::CaptureLibraryNode {
|
.send_async(ProviderMessage::new(ProviderCommand::CaptureLibraryNode {
|
||||||
path,
|
path,
|
||||||
name,
|
name,
|
||||||
|
download,
|
||||||
result_tx,
|
result_tx,
|
||||||
}))
|
}))
|
||||||
.await
|
.await
|
||||||
|
|
@ -413,9 +420,11 @@ impl CrabidyService for RpcService {
|
||||||
Err(err @ (CaptureError::InvalidName(_) | CaptureError::BadSource(_))) => {
|
Err(err @ (CaptureError::InvalidName(_) | CaptureError::BadSource(_))) => {
|
||||||
Err(Status::invalid_argument(err.to_string()))
|
Err(Status::invalid_argument(err.to_string()))
|
||||||
}
|
}
|
||||||
Err(err @ (CaptureError::TooLarge(_) | CaptureError::Disabled)) => {
|
Err(
|
||||||
Err(Status::failed_precondition(err.to_string()))
|
err @ (CaptureError::TooLarge(_)
|
||||||
}
|
| CaptureError::Disabled
|
||||||
|
| CaptureError::Unsupported),
|
||||||
|
) => Err(Status::failed_precondition(err.to_string())),
|
||||||
Err(err) => {
|
Err(err) => {
|
||||||
error!("capture_library_node failed: {err}");
|
error!("capture_library_node failed: {err}");
|
||||||
Err(Status::internal("cannot capture the subtree"))
|
Err(Status::internal("cannot capture the subtree"))
|
||||||
|
|
|
||||||
|
|
@ -201,6 +201,22 @@ impl TrackFile {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Like [`Self::from_track`], but the playable is a local
|
||||||
|
/// [`Playable::File`] at `file` instead of a link — used by download
|
||||||
|
/// captures, where the audio sits next to the toml and `file` is its
|
||||||
|
/// relative name (see `architecture/captures.md` D3). Relative paths
|
||||||
|
/// resolve against the track file's directory at play time, so the
|
||||||
|
/// capture folder stays relocatable.
|
||||||
|
pub fn from_track_with_file(track: &Track, file: &Path) -> Self {
|
||||||
|
let mut this = Self::from_track(track);
|
||||||
|
this.playable = PlayableSpec {
|
||||||
|
file: Some(file.to_path_buf()),
|
||||||
|
url: None,
|
||||||
|
link: None,
|
||||||
|
};
|
||||||
|
this
|
||||||
|
}
|
||||||
|
|
||||||
/// Serializes this file to TOML text.
|
/// Serializes this file to TOML text.
|
||||||
///
|
///
|
||||||
/// Only fails when TOML cannot represent the value
|
/// Only fails when TOML cannot represent the value
|
||||||
|
|
@ -482,6 +498,7 @@ impl Client {
|
||||||
tracks,
|
tracks,
|
||||||
is_queable: true,
|
is_queable: true,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -629,6 +646,7 @@ impl ProviderClient for Client {
|
||||||
tracks: Vec::new(),
|
tracks: Vec::new(),
|
||||||
is_queable: true,
|
is_queable: true,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -1158,6 +1176,51 @@ mod tests {
|
||||||
assert_eq!(restored, track);
|
assert_eq!(restored, track);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn from_track_with_file_plays_the_relative_sibling() {
|
||||||
|
let track = Track {
|
||||||
|
path: "/tidal/artists/1/2/3".to_string(),
|
||||||
|
artist: "Queen".to_string(),
|
||||||
|
title: "One".to_string(),
|
||||||
|
duration: Some(90),
|
||||||
|
album: Some(Album {
|
||||||
|
title: "Greatest".to_string(),
|
||||||
|
release_date: None,
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
let file = TrackFile::from_track_with_file(&track, Path::new("0001 One.flac"));
|
||||||
|
let toml_text = file.to_toml().expect("serialize");
|
||||||
|
let reparsed = TrackFile::parse(&toml_text).expect("reparse");
|
||||||
|
assert_eq!(
|
||||||
|
reparsed.playable().expect("playable"),
|
||||||
|
Playable::File("0001 One.flac".into())
|
||||||
|
);
|
||||||
|
// File playables keep the *library* path (no link rewrite), and the
|
||||||
|
// metadata survives the round trip.
|
||||||
|
let lib_path = "/captures/faves/0001%20One.cbd-track.toml";
|
||||||
|
let listed = reparsed.to_track(lib_path);
|
||||||
|
assert_eq!(listed.path, lib_path);
|
||||||
|
assert_eq!(listed.title, track.title);
|
||||||
|
assert_eq!(listed.album, track.album);
|
||||||
|
|
||||||
|
// End to end: a capture folder replays through an instance and the
|
||||||
|
// audio resolves to the absolute sibling path.
|
||||||
|
let dir = TempDir::new().expect("tempdir");
|
||||||
|
let faves = dir.path().join("faves");
|
||||||
|
fs::create_dir_all(&faves).expect("mkdir");
|
||||||
|
fs::write(faves.join("0001 One.flac"), b"flac").expect("audio");
|
||||||
|
fs::write(faves.join(track_file_name(0, "One")), toml_text).expect("toml");
|
||||||
|
let client = Client::new("/captures", dir.path().to_path_buf()).expect("instance");
|
||||||
|
let urls = client
|
||||||
|
.get_urls_for_track("/captures/faves/0001%20One.cbd-track.toml")
|
||||||
|
.await
|
||||||
|
.expect("resolve");
|
||||||
|
assert_eq!(
|
||||||
|
urls,
|
||||||
|
vec![faves.join("0001 One.flac").display().to_string()]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn from_track_serializes_sparse_metadata() {
|
fn from_track_serializes_sparse_metadata() {
|
||||||
// No artist/duration/album: serialization must not fail on `None`
|
// No artist/duration/album: serialization must not fail on `None`
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,54 @@
|
||||||
|
# Plan: captures
|
||||||
|
|
||||||
|
Ordered tasks; each names its verification (tests in `fsdy/src/lib.rs`,
|
||||||
|
`crabidy-server/src/capture.rs`, `crabidy-server/src/capture_store.rs`
|
||||||
|
and/or gates in `quality/captures.md`). Stubs, the proto field, tidal's
|
||||||
|
flag rule, and the full orchestrator/TUI plumbing exist; the new tests
|
||||||
|
fail on `todo!()` at plan time.
|
||||||
|
|
||||||
|
- [x] **T1 — fsdy file-playable constructor.**
|
||||||
|
`TrackFile::from_track_with_file`: metadata like `from_track`, playable
|
||||||
|
`file = <relative sibling>`. Verifies:
|
||||||
|
`from_track_with_file_plays_the_relative_sibling`.
|
||||||
|
- [x] **T2 — capture.rs naming + extension helpers.** `audio_file_name`
|
||||||
|
(shared `ordered_name` semantics via a public fsdy seam) and
|
||||||
|
`extension_for` (Content-Type map, URL-path fallback, `bin`).
|
||||||
|
Verifies: `extension_prefers_content_type_then_url_then_bin`,
|
||||||
|
`audio_files_pair_with_their_toml_names`.
|
||||||
|
- [x] **T3 — shared walk.** Move the bookmark walk into
|
||||||
|
`capture::capture_into`/`write_tree` parameterized by `Caps` and
|
||||||
|
`Sink`; per-track writes go through `write_track` (`Sink::Link` =
|
||||||
|
today's link file). Rewire `BookmarkStore` onto it (delete its copy;
|
||||||
|
keep its public API and `BOOKMARK_CAPS`-equivalent behavior).
|
||||||
|
Verifies: all existing `bookmark_store::tests` unchanged; gates
|
||||||
|
"Shared walk".
|
||||||
|
- [x] **T4 — download sink.** `Downloader::new` (connect timeout),
|
||||||
|
`Sink::Download` in `write_track`: per-track deadline around URL
|
||||||
|
fetch, GET, and the streamed body against `bytes_left`,
|
||||||
|
`error_for_status`, extension from the response, audio first then toml
|
||||||
|
(`from_track_with_file`). Verifies:
|
||||||
|
`download_capture_writes_audio_next_to_pointing_tomls`,
|
||||||
|
`download_capture_is_all_or_nothing`,
|
||||||
|
`download_capture_enforces_its_caps`; gates "Download sink".
|
||||||
|
- [x] **T5 — CaptureStore.** `open` (create dir + build downloader),
|
||||||
|
`capture` → blessing check (`source_allows_download`: node or track
|
||||||
|
parent) then `capture_with_caps(DOWNLOAD_CAPS)` → `capture_into`.
|
||||||
|
Verifies: `capture_requires_the_root_download_blessing`,
|
||||||
|
`capturing_a_single_blessed_track_writes_one_pair`,
|
||||||
|
`capture_validates_names_and_overwrites`; gates "Opt-in".
|
||||||
|
- [x] **T6 — TUI tests.** Binding test for `W` (Library scope, shift),
|
||||||
|
`selected_downloadable` gating (queueable-but-not-downloadable stays
|
||||||
|
closed; tracks inherit the node flag), overlay label `capture`,
|
||||||
|
submit carries `download: true`. Verifies: new `cbd-tui` tests; gates
|
||||||
|
"TUI".
|
||||||
|
- [x] **T7 — full verification.** Whole workspace suite green;
|
||||||
|
clippy/fmt/taplo/markdownlint clean; walk every gate in
|
||||||
|
`quality/captures.md` and tick it; no `todo!()` left.
|
||||||
|
- [x] **T8 — live smoke test.** Capture a real Tidal track with download
|
||||||
|
through the provider layer into a temp store (a single track keeps the
|
||||||
|
probe cheap; the multi-track walk is unit-covered), verify the audio
|
||||||
|
exists with a plausible size, replay through a `/captures` instance,
|
||||||
|
and play-resolve it — remove any temporary probe afterwards.
|
||||||
|
- [x] **T9 — docs.** `plan/summary.md` section incl. deviations;
|
||||||
|
reconcile `architecture/captures.md` (tidal flag rule, anything else
|
||||||
|
that moved).
|
||||||
|
|
@ -1,5 +1,52 @@
|
||||||
# Implementation summaries
|
# Implementation summaries
|
||||||
|
|
||||||
|
## captures (2026-07-21)
|
||||||
|
|
||||||
|
Built per `plan/captures.md`: `W` (shift) on a downloadable library
|
||||||
|
selection captures the subtree like a bookmark, but into
|
||||||
|
`<config>/crabidy/captures/<name>/` with every track's audio
|
||||||
|
**downloaded** next to its order-prefixed toml — the toml's playable is
|
||||||
|
the audio file's *relative* name (`TrackFile::from_track_with_file`), so
|
||||||
|
a capture plays with no provider round trip and the folder stays
|
||||||
|
relocatable. `/captures` is a fourth `fsdy` instance (editable top
|
||||||
|
level, nothing reserved): browse, queue, rename (`e`), delete (`d`),
|
||||||
|
and re-capture to refresh; the audio files are invisible to listings
|
||||||
|
(only dirs and `*.cbd-track.toml` count).
|
||||||
|
|
||||||
|
The bookmark walk was extracted into `capture.rs`
|
||||||
|
(`capture_into`/`write_tree`, `Caps`, one `CaptureError` for both
|
||||||
|
stores) parameterized by a per-track `Sink` — `Link` is byte-identical
|
||||||
|
bookmark behavior, `Download` fetches the first `get_urls_for_track`
|
||||||
|
URL through one shared reqwest client (30 s connect timeout, 600 s
|
||||||
|
per-track deadline, no retries), streams the body to disk against a
|
||||||
|
capture-wide byte budget, picks the extension from `Content-Type` (URL
|
||||||
|
path, then `bin`, as fallbacks), and writes the toml only after the
|
||||||
|
audio succeeded. Download caps: 1 000 dirs, 500 tracks, 4 GiB. Still
|
||||||
|
all-or-nothing with temp cleanup; downloads are sequential inside the
|
||||||
|
one spawned capture task. Download error messages carry the track's
|
||||||
|
library path, never the stream URL (`reqwest::Error::without_url`).
|
||||||
|
|
||||||
|
Nodes opt in via new additive proto flags
|
||||||
|
(`LibraryNode.is_downloadable = 8`, `LibraryNodeChild = 7`). Tidal sets
|
||||||
|
them centrally at the end of `get_lib_node`: downloadable = queueable
|
||||||
|
**or lists tracks** (so search-term track results are downloadable even
|
||||||
|
though the term node isn't queueable); children mirror `is_queable`;
|
||||||
|
tracks inherit their node's flag in the TUI. The server re-enforces at
|
||||||
|
the capture root (`Unsupported` → `failed_precondition`); the rpc
|
||||||
|
gained `CaptureLibraryNodeRequest.download = 3` (additive; old clients
|
||||||
|
keep bookmarking).
|
||||||
|
|
||||||
|
Deviations from the plan/architecture: the naming helper ended up
|
||||||
|
`audio_file_name` (the path variant was clippy-dead); the tidal flag
|
||||||
|
rule grew the "or lists tracks" clause (architecture D4 reconciled);
|
||||||
|
the live probe downloaded a single real track (8.6 MB m4a,
|
||||||
|
Content-Type-derived extension, replayed through a `/captures`
|
||||||
|
instance) instead of a whole album — the multi-track walk is
|
||||||
|
unit-covered and a full album download is needlessly heavy for a smoke
|
||||||
|
test. 142 workspace tests green (1 new in `fsdy`, 8 in
|
||||||
|
`capture`/`capture_store`, 3 TUI + 1 extended); every gate in
|
||||||
|
`quality/captures.md` checked.
|
||||||
|
|
||||||
## bookmarks (2026-07-21)
|
## bookmarks (2026-07-21)
|
||||||
|
|
||||||
Built per `plan/bookmarks.md`: `w` on a queueable library selection now
|
Built per `plan/bookmarks.md`: `w` on a queueable library selection now
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,84 @@
|
||||||
|
# Quality gates: captures
|
||||||
|
|
||||||
|
Criteria the implementation must satisfy beyond the automatic tests
|
||||||
|
(`crabidy-server/src/capture.rs`, `crabidy-server/src/capture_store.rs`,
|
||||||
|
`fsdy/src/lib.rs`, plus the TUI tests added during implementation). Each
|
||||||
|
gate is pass/fail by reading the code.
|
||||||
|
|
||||||
|
## Shared walk (refactor)
|
||||||
|
|
||||||
|
- [x] Bookmarks and download captures run through **one** walk
|
||||||
|
(`capture::capture_into`/`write_tree`); `bookmark_store` keeps no copy
|
||||||
|
of the worklist, caps, tmp-and-swap, or cleanup logic.
|
||||||
|
- [x] The refactor is behavior-preserving for bookmarks: every existing
|
||||||
|
`bookmark_store` test passes unchanged (module path of `CaptureError`
|
||||||
|
aside).
|
||||||
|
- [x] The walk stays iterative (worklist), all-or-nothing, and removes the
|
||||||
|
temp folder on every failure path — including failed downloads and the
|
||||||
|
byte budget.
|
||||||
|
|
||||||
|
## Download sink
|
||||||
|
|
||||||
|
- [x] Every external call is bounded: connect timeout on the shared HTTP
|
||||||
|
client and one per-track deadline covering URL fetch, request, and the
|
||||||
|
whole body stream. No retries.
|
||||||
|
- [x] Bodies are **streamed** to disk (never buffered whole) and counted
|
||||||
|
against the capture's byte budget while streaming; exceeding it is
|
||||||
|
`TooLarge`, not partial data left behind.
|
||||||
|
- [x] Non-2xx responses and transport errors are typed
|
||||||
|
(`CaptureError::Download`) — no panic on any network condition.
|
||||||
|
- [x] Download log lines carry paths, names, and counts — never stream
|
||||||
|
URLs (they embed tokens) and never file contents.
|
||||||
|
- [x] The toml is written only after its audio file succeeded, with a
|
||||||
|
relative `file` playable naming the sibling
|
||||||
|
(`TrackFile::from_track_with_file`); the audio name shares the toml's
|
||||||
|
order prefix and sanitizer (`capture::audio_file_name`).
|
||||||
|
- [x] Downloads run sequentially inside the one spawned capture task; the
|
||||||
|
orchestrator loop keeps serving commands during a capture.
|
||||||
|
|
||||||
|
## Opt-in (is_downloadable)
|
||||||
|
|
||||||
|
- [x] Tidal sets the flag centrally: nodes are downloadable when queueable
|
||||||
|
or when they list tracks; children mirror `is_queable`. No per-arm
|
||||||
|
copies to drift.
|
||||||
|
- [x] Every other provider (fs, queues, bookmarks, captures, orchestrator
|
||||||
|
root) leaves the flag `false` — a capture can never be built from
|
||||||
|
another capture's or bookmark's links masquerading as sources.
|
||||||
|
- [x] The server enforces the blessing at the capture root (directory
|
||||||
|
source: its own node; track source: its parent node) with
|
||||||
|
`CaptureError::Unsupported`; an unreadable root is `BadSource`.
|
||||||
|
|
||||||
|
## RPC and orchestrator
|
||||||
|
|
||||||
|
- [x] `CaptureLibraryNodeRequest.download` is additive: old clients (no
|
||||||
|
flag) keep getting bookmarks, byte-for-byte.
|
||||||
|
- [x] Error mapping: `InvalidName`/`BadSource` → `invalid_argument`;
|
||||||
|
`TooLarge`/`Disabled`/`Unsupported` → `failed_precondition`;
|
||||||
|
download/walk/write failures → `internal`.
|
||||||
|
- [x] The orchestrator routes `/captures` in every `ProviderClient` method
|
||||||
|
(same completeness as `/bookmarks`), and `get_lib_root` lists the
|
||||||
|
`captures` child only when the instance exists.
|
||||||
|
- [x] Captures init is non-fatal: no config dir or an unopenable store
|
||||||
|
disables download captures and the `/captures` mount, never the server
|
||||||
|
(and never bookmarks).
|
||||||
|
- [x] `/captures` mounts with an editable top level (no reserved names):
|
||||||
|
captures are renamable (`e`) and deletable (`d`) like bookmarks.
|
||||||
|
|
||||||
|
## TUI
|
||||||
|
|
||||||
|
- [x] `W` (shift) is bound in `Scope::Library` with a help description and
|
||||||
|
passes the bindings-table invariant tests unchanged; `w` behavior is
|
||||||
|
untouched.
|
||||||
|
- [x] The capture overlay opens for `W` only when the bare selection is
|
||||||
|
queueable **and** downloadable; tracks inherit their node's flag;
|
||||||
|
marks are ignored. Label: `capture` (vs `bookmark`).
|
||||||
|
- [x] `MessageFromUi::CaptureNode` carries `download`; a failed capture is
|
||||||
|
logged and never tears down the poll loop.
|
||||||
|
|
||||||
|
## Hygiene
|
||||||
|
|
||||||
|
- [x] New public items are documented; docs state error/edge behavior.
|
||||||
|
- [x] `clippy -D warnings`, `fmt`, `taplo`, `markdownlint` clean on the
|
||||||
|
whole workspace; all tests green.
|
||||||
|
- [x] `architecture/captures.md` reconciled where the implementation
|
||||||
|
diverged (e.g. the exact tidal flag rule).
|
||||||
|
|
@ -128,6 +128,7 @@ impl crabidy_core::ProviderClient for Client {
|
||||||
],
|
],
|
||||||
is_queable: false,
|
is_queable: false,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -155,6 +156,7 @@ impl crabidy_core::ProviderClient for Client {
|
||||||
children: Vec::new(),
|
children: Vec::new(),
|
||||||
is_queable: false,
|
is_queable: false,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
};
|
};
|
||||||
let user_id = user_id.ok_or(crabidy_core::ProviderError::UnknownUser)?;
|
let user_id = user_id.ok_or(crabidy_core::ProviderError::UnknownUser)?;
|
||||||
for playlist in self
|
for playlist in self
|
||||||
|
|
@ -186,6 +188,7 @@ impl crabidy_core::ProviderClient for Client {
|
||||||
children: Vec::new(),
|
children: Vec::new(),
|
||||||
is_queable: true,
|
is_queable: true,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
TidalPath::Artists => {
|
TidalPath::Artists => {
|
||||||
|
|
@ -197,6 +200,7 @@ impl crabidy_core::ProviderClient for Client {
|
||||||
children: Vec::new(),
|
children: Vec::new(),
|
||||||
is_queable: false,
|
is_queable: false,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
};
|
};
|
||||||
let user_id = user_id.ok_or(crabidy_core::ProviderError::UnknownUser)?;
|
let user_id = user_id.ok_or(crabidy_core::ProviderError::UnknownUser)?;
|
||||||
for artist in self.get_users_artists(&user_id).await? {
|
for artist in self.get_users_artists(&user_id).await? {
|
||||||
|
|
@ -231,6 +235,7 @@ impl crabidy_core::ProviderClient for Client {
|
||||||
children,
|
children,
|
||||||
is_queable: true,
|
is_queable: true,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
TidalPath::Album { album, .. } => {
|
TidalPath::Album { album, .. } => {
|
||||||
|
|
@ -249,6 +254,7 @@ impl crabidy_core::ProviderClient for Client {
|
||||||
children: Vec::new(),
|
children: Vec::new(),
|
||||||
is_queable: true,
|
is_queable: true,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
TidalPath::Search => crabidy_core::proto::crabidy::LibraryNode {
|
TidalPath::Search => crabidy_core::proto::crabidy::LibraryNode {
|
||||||
|
|
@ -275,6 +281,7 @@ impl crabidy_core::ProviderClient for Client {
|
||||||
.collect(),
|
.collect(),
|
||||||
is_queable: false,
|
is_queable: false,
|
||||||
is_creatable: true,
|
is_creatable: true,
|
||||||
|
is_downloadable: false,
|
||||||
},
|
},
|
||||||
TidalPath::SearchTerm(encoded) => {
|
TidalPath::SearchTerm(encoded) => {
|
||||||
let term = crabidy_core::decode_segment(encoded);
|
let term = crabidy_core::decode_segment(encoded);
|
||||||
|
|
@ -290,6 +297,16 @@ impl crabidy_core::ProviderClient for Client {
|
||||||
return Err(crabidy_core::ProviderError::MalformedPath);
|
return Err(crabidy_core::ProviderError::MalformedPath);
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
// Tidal implements download captures (architecture/captures.md D4):
|
||||||
|
// every queueable subtree may be captured with `W`, and every node
|
||||||
|
// that lists tracks blesses them for download (listed tracks
|
||||||
|
// inherit their node's flag — this covers search-term results,
|
||||||
|
// whose node is not queueable as a whole).
|
||||||
|
let mut node = node;
|
||||||
|
node.is_downloadable = node.is_queable || !node.tracks.is_empty();
|
||||||
|
for child in &mut node.children {
|
||||||
|
child.is_downloadable = child.is_queable;
|
||||||
|
}
|
||||||
Ok(node)
|
Ok(node)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -897,6 +914,7 @@ impl Client {
|
||||||
children,
|
children,
|
||||||
is_queable: false,
|
is_queable: false,
|
||||||
is_creatable: false,
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue