Allow W captures of queues and bookmarks

The /queues and /bookmarks instances now advertise is_downloadable on
every node (new fsdy with_downloadable_nodes option). Because such
captures mix providers, the download sink skips tracks whose source
cannot be captured (unresolvable streams, local file playables) with a
warning instead of aborting; real download failures stay fatal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Test User 2026-07-21 14:05:43 +02:00
parent c6e9f71cf4
commit 8032bec8e1
6 changed files with 143 additions and 17 deletions

View File

@ -90,8 +90,14 @@ capture must not fill the disk. Downloads run sequentially (gentle on the
provider, trivially bounded memory); the whole capture already runs on a
spawned task, so the orchestrator keeps serving.
All-or-nothing is kept: any failed download aborts the capture and removes
the temp folder. A capture that lists 30 tracks *has* 30 playable files.
All-or-nothing is kept for real failures: any failed download (bad
status, transport error, timeout) aborts the capture and removes the temp
folder. The one softening: a track whose source **cannot be captured at
all** — its stream fails to resolve, or resolves to a non-http(s) target
(a local file playable) — is *skipped* with a warning instead of
aborting. Queue and bookmark captures mix providers (D4), and one local
`/fs` entry must not kill the downloadable rest; the skipped track simply
has no pair in the capture.
### D4 — Nodes opt in via `is_downloadable`
@ -100,10 +106,13 @@ New proto fields `LibraryNode.is_downloadable = 8` and
centrally at the end of `get_lib_node`: a node is downloadable when it is
**queueable or lists tracks** (the "or lists tracks" covers search-term
result nodes, which are not queueable as a whole but whose track results
are downloadable); children mirror `is_queable`. Everything else (fs,
queues, bookmarks, captures, orchestrator roots) leaves the default
`false`; capturing a capture is pointless and links must not masquerade
as downloads.
are downloadable); children mirror `is_queable`. The `/queues` and
`/bookmarks` instances opt in wholesale
(`fsdy::Client::with_downloadable_nodes`): their entries are links into
downloadable providers, so `W` on a saved queue or bookmark downloads
its resolvable tracks and skips the rest (D3). `/fs` and `/captures`
stay `false` — capturing a capture is pointless, and local trees have
nothing to download.
Tracks carry no flag: a listed track inherits its containing node's
`is_downloadable` (TUI) — a Tidal album's tracks are downloadable because

View File

@ -11,6 +11,7 @@ use std::path::{Path, PathBuf};
use crabidy_core::proto::crabidy::Track;
use crabidy_core::ProviderClient;
use tracing::warn;
/// Connect timeout for download requests.
pub const DOWNLOAD_CONNECT_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
@ -141,6 +142,13 @@ impl Downloader {
/// The unbounded body of [`Self::download_track`]: resolve the stream
/// URL, stream the response to disk against the byte budget, then
/// write the toml.
///
/// A track whose source cannot be captured — the stream fails to
/// resolve, or it resolves to something other than an http(s) URL
/// (e.g. a local file playable) — is **skipped** with a warning
/// instead of aborting the capture (architecture/captures.md D3):
/// queue and bookmark captures mix providers, and one local track
/// must not kill the rest. Actual download failures stay fatal.
async fn fetch_track<C>(
&self,
client: &C,
@ -153,13 +161,23 @@ impl Downloader {
C: ProviderClient + Sync,
{
let track_path = track.path.as_str();
let urls = client
.get_urls_for_track(track_path)
.await
.map_err(|err| CaptureError::BadSource(format!("{track_path}: {err}")))?;
let url = urls
.first()
.ok_or_else(|| CaptureError::BadSource(format!("{track_path}: no stream url")))?;
let urls = match client.get_urls_for_track(track_path).await {
Ok(urls) => urls,
Err(err) => {
warn!(path = track_path, "skipping uncapturable track: {err}");
return Ok(());
}
};
let Some(url) = urls.first() else {
warn!(path = track_path, "skipping track without a stream url");
return Ok(());
};
if !(url.starts_with("http://") || url.starts_with("https://")) {
// Local file playables (fs tracks, existing captures) resolve
// to paths, not URLs — nothing to download.
warn!(path = track_path, "skipping non-http playable");
return Ok(());
}
let download_err = |err: reqwest::Error| {
CaptureError::Download(format!("{track_path}: {}", err.without_url()))
};

View File

@ -413,6 +413,46 @@ mod tests {
assert_eq!(fs::read_dir(store.dir()).expect("store dir").count(), 0);
}
#[tokio::test]
async fn download_capture_skips_uncapturable_tracks() {
// A queue-like source: a downloadable fsdy instance whose folder
// mixes an http track with a local-file track — like a persisted
// queue linking tidal and fs entries.
let url = serve("200 OK", "audio/flac", b"flacbytes".to_vec()).await;
let src = TempDir::new().expect("source tempdir");
let mix = src.path().join("mix");
fs::create_dir_all(&mix).expect("mkdir");
fs::write(
mix.join("01 web.cbd-track.toml"),
format!("title = \"web\"\n[playable]\nurl = {url:?}\n"),
)
.expect("write");
fs::write(mix.join("audio.flac"), b"local").expect("write");
fs::write(
mix.join("02 local.cbd-track.toml"),
"title = \"local\"\n[playable]\nfile = \"audio.flac\"\n",
)
.expect("write");
let source = fsdy::Client::new("/queues", src.path().to_path_buf())
.expect("source instance")
.with_downloadable_nodes();
let (store, _dir) = store().await;
store
.capture(&source, "/queues/mix", "mixed")
.await
.expect("capture succeeds despite the local track");
// The http track downloaded; the local-file track was skipped
// (no pair, no abort), keeping its listing position's prefix.
assert_eq!(
visible(&store.dir().join("mixed")),
vec![
"0001 web.cbd-track.toml".to_string(),
"0001 web.flac".into()
]
);
}
#[tokio::test]
async fn capture_validates_names_and_overwrites() {
let url = serve("200 OK", "audio/flac", b"x".to_vec()).await;

View File

@ -274,7 +274,9 @@ impl ProviderClient for ProviderOrchestrator {
let queues_client = match crabidy_server::queue_store::queues_dir() {
Some(dir) => match fsdy::Client::new(QUEUES_PROVIDER_ROOT, dir) {
Ok(client) => Some(Arc::new(
client.with_editable_top_level(&[CURRENT_QUEUE_NAME]),
client
.with_editable_top_level(&[CURRENT_QUEUE_NAME])
.with_downloadable_nodes(),
)),
Err(err) => {
warn!("queues library disabled: {err}");
@ -304,7 +306,11 @@ impl ProviderClient for ProviderOrchestrator {
};
let bookmarks_client = bookmark_store.as_ref().and_then(|store| {
match fsdy::Client::new(BOOKMARKS_PROVIDER_ROOT, store.dir().to_path_buf()) {
Ok(client) => Some(Arc::new(client.with_editable_top_level(&[]))),
Ok(client) => Some(Arc::new(
client
.with_editable_top_level(&[])
.with_downloadable_nodes(),
)),
Err(err) => {
warn!("bookmarks library disabled: {err}");
None

View File

@ -315,6 +315,12 @@ pub struct Client {
/// Folder names exempt from top-level editing (e.g. the auto-persisted
/// `current` queue) and rejected as rename targets.
reserved: Vec<String>,
/// Whether this instance's nodes advertise download captures
/// (`is_downloadable`, architecture/captures.md D4) — on for
/// `/queues` and `/bookmarks`, whose entries mostly link to
/// downloadable providers; off for `/fs` and `/captures`. Tracks
/// whose source cannot be captured are skipped by the capture walk.
downloadable_nodes: bool,
}
impl Client {
@ -344,6 +350,7 @@ impl Client {
provider_root: provider_root.to_string(),
editable_top_level: false,
reserved: Vec::new(),
downloadable_nodes: false,
})
}
@ -359,6 +366,16 @@ impl Client {
self
}
/// Marks every node of this instance downloadable (`is_downloadable`
/// — `W` captures the subtree, see architecture/captures.md D4). Used
/// by `/queues` and `/bookmarks`, whose entries are links into
/// downloadable providers; the capture walk skips tracks whose
/// source cannot be captured (local files, unresolvable links).
pub fn with_downloadable_nodes(mut self) -> Self {
self.downloadable_nodes = true;
self
}
/// The provider root this instance owns (e.g. `/fs`), as configured
/// through [`Self::new`].
pub fn provider_root(&self) -> &str {
@ -478,6 +495,7 @@ impl Client {
let mut child = LibraryNodeChild::new(child_lib, name, true);
child.is_editable = editable;
child.is_deletable = editable;
child.is_downloadable = self.downloadable_nodes;
child
})
.collect();
@ -498,7 +516,7 @@ impl Client {
tracks,
is_queable: true,
is_creatable: false,
is_downloadable: false,
is_downloadable: self.downloadable_nodes,
})
}
@ -646,7 +664,7 @@ impl ProviderClient for Client {
tracks: Vec::new(),
is_queable: true,
is_creatable: false,
is_downloadable: false,
is_downloadable: self.downloadable_nodes,
}
}
@ -1237,6 +1255,26 @@ mod tests {
assert_eq!(reparsed.to_track("/queues/x.cbd-track.toml"), track);
}
#[tokio::test]
async fn downloadable_instances_flag_every_node() {
let dir = TempDir::new().expect("tempdir");
fs::create_dir_all(dir.path().join("mix")).expect("mkdir");
// Off by default: /fs and /captures stay uncapturable.
let plain = Client::new("/fs", dir.path().to_path_buf()).expect("instance");
let node = plain.get_lib_node("/fs").await.expect("list");
assert!(!node.is_downloadable);
assert!(!node.children[0].is_downloadable);
// Opted in (/queues, /bookmarks): nodes and children advertise W.
let blessed = Client::new("/queues", dir.path().to_path_buf())
.expect("instance")
.with_downloadable_nodes();
let node = blessed.get_lib_node("/queues").await.expect("list");
assert!(node.is_downloadable);
assert!(node.children[0].is_downloadable);
let nested = blessed.get_lib_node("/queues/mix").await.expect("list");
assert!(nested.is_downloadable);
}
// ---- mutable top level (bookmarks / saved queues) ---------------------
/// An editable instance over a root with two folders (one reserved) and

View File

@ -1,5 +1,20 @@
# Implementation summaries
## captures follow-up: W on queues and bookmarks (2026-07-21)
Small fix on top of the captures feature: `/queues` and `/bookmarks`
nodes are now `W`-capturable. `fsdy::Client` gained
`with_downloadable_nodes()` (instance-wide `is_downloadable`, applied to
the queues and bookmarks mounts; `/fs` and `/captures` stay off), and
the download sink softens all-or-nothing for exactly one case: a track
whose source cannot be captured — stream resolution fails, or resolves
to a non-http(s) target like a local file playable — is skipped with a
warning instead of aborting, since queue/bookmark captures mix
providers. Real download failures (bad status, transport, timeout) stay
fatal. `architecture/captures.md` D3/D4 reconciled; new tests
`downloadable_instances_flag_every_node` (fsdy) and
`download_capture_skips_uncapturable_tracks` (capture store).
## youtube-provider (2026-07-21)
Built per `plan/youtube-provider.md`: a new workspace crate **`ytdy`**