Add a YouTube provider backed by yt-dlp

The new ytdy crate mounts /youtube: login-free search terms exactly
like tidal's (creatable, renamable, deletable, results as queueable
downloadable tracks) and, when a cookies file is configured, the
user's playlists. All extraction runs through one bounded subprocess
seam (argv-only, per-call timeout, stdout cap, typed errors) so tests
drive the provider with a fake script. yt-dlp is declared in devenv;
a failed binary probe disables the provider, never the server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Test User 2026-07-21 14:01:58 +02:00
parent 7f0e900c56
commit c6e9f71cf4
13 changed files with 1436 additions and 0 deletions

16
Cargo.lock generated
View File

@ -760,6 +760,7 @@ dependencies = [
"tracing", "tracing",
"tracing-appender", "tracing-appender",
"tracing-subscriber", "tracing-subscriber",
"ytdy",
] ]
[[package]] [[package]]
@ -5133,6 +5134,21 @@ dependencies = [
"synstructure", "synstructure",
] ]
[[package]]
name = "ytdy"
version = "0.1.0"
dependencies = [
"async-trait",
"crabidy-core",
"serde",
"serde_json",
"tempfile",
"thiserror 2.0.19",
"tokio",
"toml",
"tracing",
]
[[package]] [[package]]
name = "zbus" name = "zbus"
version = "5.18.0" version = "5.18.0"

View File

@ -7,6 +7,7 @@ members = [
"crabidy-server", "crabidy-server",
"fsdy", "fsdy",
"tidaldy", "tidaldy",
"ytdy",
] ]
[workspace.package] [workspace.package]
@ -68,3 +69,4 @@ audio-player = { path = "audio-player" }
crabidy-core = { path = "crabidy-core" } crabidy-core = { path = "crabidy-core" }
fsdy = { path = "fsdy" } fsdy = { path = "fsdy" }
tidaldy = { path = "tidaldy" } tidaldy = { path = "tidaldy" }
ytdy = { path = "ytdy" }

View File

@ -0,0 +1,186 @@
# YouTube provider (ytdy)
## Context and problem statement
A new library provider for YouTube, mounted at `/youtube`:
- **Search** works without any login — creatable search-term nodes like
`/tidal/search` (`%` creates a term, results list as tracks).
- **Login is optional.** When configured, the user's **playlists** appear
as an extra subtree; without it, the provider still works (search
only).
- **Captures** (`W`, download) must work on YouTube nodes — search
results and playlists are downloadable.
## Assumptions (confirmed)
- The captures machinery is provider-agnostic: anything that answers
`get_lib_node`/`get_urls_for_track` and raises `is_downloadable` gets
`w`/`W` for free — no new wire or TUI work at all.
- The TUI's creatable/editable/deletable node flows (`%`/`e`/`d`) are
generic; mirroring tidal's search-term semantics costs no TUI change.
- `pkgs.yt-dlp` (2026.06.09) exists in nixpkgs; the devenv already pins
the toolchain, so the engine binary is declared, not assumed.
- The audio player streams plain https URLs; googlevideo stream URLs
(from `bestaudio`) are plain https and also downloadable with the
captures reqwest client.
## Decisions
### D1 — Extraction engine: `yt-dlp` subprocess
Options considered:
- *(a)* **Pure-Rust extractor crates** (`rustypipe`, `rusty_ytdl`): no
external binary, but they chase YouTube's extraction changes with
small maintainer teams, and logged-in user playlists are weakly or not
supported.
- *(b)* **Invidious/Piped instances**: no extraction code at all, but a
hard runtime dependency on third-party servers of unpredictable
availability — worse than a local binary for a self-hosted player.
- *(c)* **`yt-dlp` as a subprocess** with `-J` JSON output: the de-facto
standard extractor, fastest to track YouTube changes, supports search
(`ytsearchN:`), playlists, cookies-based login, and direct stream
URLs. Cost: a non-Rust runtime dependency and subprocess plumbing.
**Decision: (c).** The binary is declared in `devenv.nix` (dev) and is a
documented runtime requirement (deploys). All calls go through one
`Engine` seam (`tokio::process::Command`, `kill_on_drop`, per-call
timeout, bounded stdout, `serde_json` parsing) so tests fake the binary
with a script and a future pure-Rust engine stays swappable.
### D2 — Crate `ytdy`, mounted at `/youtube`, non-fatal init
New workspace crate `ytdy` implementing `ProviderClient`, following
`tidaldy`'s shape. `init` probes `<binary> --version` (with timeout);
a missing or broken binary disables the provider with a warning — the
server and every other provider keep running. Settings (`ytdy.toml`):
```toml
binary = "yt-dlp" # optional; PATH lookup by default
cookies = "/path/cookies.txt" # optional; presence = "logged in"
search_results = 20 # ytsearchN cap
```
The orchestrator gains `youtube_client: Option<Arc<ytdy::Client>>` and
`/youtube` routing arms (same completeness as the other providers);
`get_lib_root` lists `youtube` only when the probe succeeded.
### D3 — Tree shape
- `/youtube` — children: `search` (always, `is_creatable`), `playlists`
(only when cookies are configured).
- `/youtube/search/<term>` — created via `%` like tidal search; terms
live in memory (`RwLock<Vec<String>>`, dedup, recreated implicitly on
stale paths), renamable and deletable. The node lists the top
`search_results` results as **tracks** (`ytsearchN:<term>`,
`--flat-playlist`); it is queueable (results are homogeneous tracks,
unlike tidal's mixed search).
- `/youtube/playlists` — the user's playlists as children
(`https://www.youtube.com/feed/playlists` with cookies, flat).
- `/youtube/playlists/<pid>` — playlist entries as tracks.
- Track paths: `<node>/<videoid>`; metadata from the flat entries
(title, uploader as artist, duration). `get_metadata_for_track` runs
a single-video `-J` when called directly.
### D4 — Streams, downloads, login
- `get_urls_for_track`: `yt-dlp -f bestaudio/best -g --no-playlist
<video>` → the stream URL(s). Cookies are passed to **every** call
when configured (search and streams benefit too, e.g. age-gated
videos).
- Downloadability mirrors tidal's central rule: a node is downloadable
when it is queueable or lists tracks; children mirror `is_queable`.
`extension_for` gains `audio/webm → webm` (bestaudio is usually opus
in webm; googlevideo URLs carry no path extension).
- "Logged in" is exactly "a cookies file is configured and readable at
init" — no OAuth flow of our own, no credential storage beyond the
user-provided file. The cookies path is config; its **contents are a
secret** and must never appear in logs or error reports.
### D5 — Subprocess discipline
- One shared `Engine` with the binary path; every invocation:
`--no-warnings -J` (or `-g`), argument-list only (never shell),
per-call timeout (default 60 s, configurable for tests), stdout
capped, stderr summarized into warnings (never echoed wholesale — it
can contain URLs).
- Errors are typed (`EngineError`: spawn/timeout/exit-status/parse) and
map to `ProviderError::FetchError`/`MalformedPath` at the trait
boundary; no panic on any subprocess condition.
- Calls run sequentially per request (no engine-level queue); the
orchestrator already spawns long walks.
### D6 — Out of scope (explicitly)
- OAuth/device login flows, cookie refresh, or storing credentials.
- Uploads, likes, subscriptions feeds, comments; YouTube Music.
- Caching search or playlist results across calls (each listing is a
fresh subprocess call, like tidal's fresh HTTP calls).
- SponsorBlock, chapters, DRM-protected content (skipped as unplayable).
## Structure
```d2
direction: right
server: crabidy-server {
orch: ProviderOrchestrator
}
ytdy: "ytdy (crate)" {
client: "Client\n(ProviderClient)"
terms: "search terms\n(in-memory, like tidal)"
engine: "Engine\n(subprocess seam)"
client -> terms
client -> engine
}
bin: "yt-dlp binary\n(devenv / PATH)" {
shape: hexagon
}
yt: "YouTube" { shape: cloud }
cookies: "cookies.txt\n(optional, user-provided)" { shape: page }
server.orch -> ytdy.client: "/youtube/..."
ytdy.engine -> bin: "argv + -J, timeout,\nbounded stdout"
bin -> yt
cookies -> ytdy.engine: "--cookies (when set)"
```
## Key flow: search, then W
```d2
shape: sequence_diagram
tui: TUI
orch: Orchestrator
y: ytdy
e: yt-dlp
tui -> orch: "% on /youtube/search: 'lofi'"
orch -> y: "create_lib_node"
y -> e: "ytsearch20:lofi -J --flat-playlist"
e -> y: "entries (id, title, uploader, duration)"
y -> tui: "term node: 20 tracks, downloadable"
tui -> orch: "W → CaptureLibraryNode(download)"
orch -> y: "per track: get_urls_for_track"
y -> e: "-f bestaudio -g <id>"
e -> y: "googlevideo url"
orch -> orch: "captures store: stream to\n/captures/<name>/ (webm + toml)"
```
## Risks and open questions
- **`feed/playlists` extraction**: the exact yt-dlp invocation for "my
playlists" remains **live-unvalidated** (no cookies were available
during development; search, streams, and download captures were
validated live). Fallback if it turns out wrong: adjust the feed URL
in `playlists_node`, or bookmark playlist URLs as search terms.
- **yt-dlp breakage/drift**: extraction can break until the devenv pin
is bumped; errors stay typed and non-fatal.
- **Stream URL lifetime**: googlevideo URLs expire (~6 h); captures
download immediately after resolving, playback resolves on play —
both within the window.
- **Terms of service**: downloads are personal-use copies of streams the
client can already play, same stance as tidal captures.

View File

@ -28,6 +28,7 @@ tonic.workspace = true
tracing.workspace = true tracing.workspace = true
tracing-appender.workspace = true tracing-appender.workspace = true
tracing-subscriber.workspace = true tracing-subscriber.workspace = true
ytdy.workspace = true
[dev-dependencies] [dev-dependencies]
tempfile.workspace = true tempfile.workspace = true

View File

@ -337,6 +337,9 @@ fn extension_for(content_type: Option<&str>, url: &str) -> String {
"audio/mpeg" | "audio/mp3" => Some("mp3"), "audio/mpeg" | "audio/mp3" => Some("mp3"),
"audio/ogg" => Some("ogg"), "audio/ogg" => Some("ogg"),
"audio/wav" | "audio/x-wav" => Some("wav"), "audio/wav" | "audio/x-wav" => Some("wav"),
// YouTube bestaudio is usually opus in webm; googlevideo
// URLs carry no path extension to fall back on.
"audio/webm" | "video/webm" => Some("webm"),
_ => None, _ => None,
}); });
match mapped.or_else(|| url_extension(url)) { match mapped.or_else(|| url_extension(url)) {
@ -379,6 +382,7 @@ mod tests {
("audio/mpeg", "mp3"), ("audio/mpeg", "mp3"),
("audio/ogg", "ogg"), ("audio/ogg", "ogg"),
("audio/wav", "wav"), ("audio/wav", "wav"),
("audio/webm", "webm"),
] { ] {
assert_eq!(extension_for(Some(ct), "https://x.test/s"), ext); assert_eq!(extension_for(Some(ct), "https://x.test/s"), ext);
} }

View File

@ -36,6 +36,9 @@ pub struct ProviderOrchestrator {
/// The download-capture writer; `None` disables download captures /// The download-capture writer; `None` disables download captures
/// (and the `/captures` mount goes with it). /// (and the `/captures` mount goes with it).
capture_store: Option<Arc<CaptureStore>>, capture_store: Option<Arc<CaptureStore>>,
/// The YouTube provider (yt-dlp backed); `None` when the binary
/// probe failed at init (architecture/youtube-provider.md D2).
youtube_client: Option<Arc<ytdy::Client>>,
} }
/// Whether a path belongs to the filesystem provider. /// Whether a path belongs to the filesystem provider.
@ -59,6 +62,11 @@ fn captures_owns(path: &str) -> bool {
path == CAPTURES_PROVIDER_ROOT || path.starts_with("/captures/") path == CAPTURES_PROVIDER_ROOT || path.starts_with("/captures/")
} }
/// Whether a path belongs to the YouTube provider.
fn youtube_owns(path: &str) -> bool {
path == ytdy::PROVIDER_ROOT || path.starts_with("/youtube/")
}
impl ProviderOrchestrator { impl ProviderOrchestrator {
/// The fs client, or `MalformedPath` (with a warning) when the /// The fs client, or `MalformedPath` (with a warning) when the
/// provider is disabled — a `/fs` path then has no owner. /// provider is disabled — a `/fs` path then has no owner.
@ -95,6 +103,15 @@ impl ProviderOrchestrator {
ProviderError::MalformedPath ProviderError::MalformedPath
}) })
} }
/// The YouTube client, or `MalformedPath` (with a warning) when the
/// provider is disabled — a `/youtube` path then has no owner.
fn youtube_provider(&self) -> Result<&ytdy::Client, ProviderError> {
self.youtube_client.as_deref().ok_or_else(|| {
warn!("youtube provider is disabled");
ProviderError::MalformedPath
})
}
pub fn run(self) { pub fn run(self) {
tokio::spawn(async move { tokio::spawn(async move {
// Behind an Arc so long-running resolves can be spawned onto // Behind an Arc so long-running resolves can be spawned onto
@ -318,6 +335,23 @@ impl ProviderClient for ProviderOrchestrator {
} }
} }
}); });
// YouTube: non-fatal like the local providers — a missing or
// broken yt-dlp binary only costs the `/youtube` subtree.
let yt_config_file = config_dir.join("ytdy.toml");
debug!(config_file = %yt_config_file.display(), "loading youtube config");
let raw_yt_settings = fs::read_to_string(&yt_config_file).unwrap_or_default();
let youtube_client = match ytdy::Client::init(&raw_yt_settings).await {
Ok(client) => {
if let Err(err) = tokio::fs::write(&yt_config_file, client.settings()).await {
error!("failed to write ytdy config file: {err}");
}
Some(Arc::new(client))
}
Err(err) => {
warn!("youtube provider disabled: {err}");
None
}
};
let (provider_tx, provider_rx) = flume::bounded(100); let (provider_tx, provider_rx) = flume::bounded(100);
Ok(Self { Ok(Self {
provider_rx, provider_rx,
@ -329,6 +363,7 @@ impl ProviderClient for ProviderOrchestrator {
bookmark_store, bookmark_store,
captures_client, captures_client,
capture_store, capture_store,
youtube_client,
}) })
} }
@ -365,6 +400,12 @@ impl ProviderClient for ProviderOrchestrator {
.as_ref() .as_ref()
.is_some_and(|captures| captures.is_track_path(path)); .is_some_and(|captures| captures.is_track_path(path));
} }
if youtube_owns(path) {
return self
.youtube_client
.as_ref()
.is_some_and(|youtube| youtube.is_track_path(path));
}
false false
} }
@ -391,6 +432,12 @@ impl ProviderClient for ProviderOrchestrator {
.get_urls_for_track(track_path) .get_urls_for_track(track_path)
.await; .await;
} }
if youtube_owns(track_path) {
return self
.youtube_provider()?
.get_urls_for_track(track_path)
.await;
}
warn!(path = track_path, "no provider owns this track path"); warn!(path = track_path, "no provider owns this track path");
Err(ProviderError::MalformedPath) Err(ProviderError::MalformedPath)
} }
@ -421,6 +468,12 @@ impl ProviderClient for ProviderOrchestrator {
.get_metadata_for_track(track_path) .get_metadata_for_track(track_path)
.await; .await;
} }
if youtube_owns(track_path) {
return self
.youtube_provider()?
.get_metadata_for_track(track_path)
.await;
}
warn!(path = track_path, "no provider owns this track path"); warn!(path = track_path, "no provider owns this track path");
Err(ProviderError::MalformedPath) Err(ProviderError::MalformedPath)
} }
@ -456,6 +509,11 @@ impl ProviderClient for ProviderOrchestrator {
); );
root_node.children.push(child); root_node.children.push(child);
} }
if self.youtube_client.is_some() {
let child =
LibraryNodeChild::new(ytdy::PROVIDER_ROOT.to_owned(), "youtube".to_owned(), false);
root_node.children.push(child);
}
root_node root_node
} }
@ -480,6 +538,9 @@ impl ProviderClient for ProviderOrchestrator {
if captures_owns(path) { if captures_owns(path) {
return self.captures_provider()?.get_lib_node(path).await; return self.captures_provider()?.get_lib_node(path).await;
} }
if youtube_owns(path) {
return self.youtube_provider()?.get_lib_node(path).await;
}
warn!(path, "no provider owns this path"); warn!(path, "no provider owns this path");
Err(ProviderError::MalformedPath) Err(ProviderError::MalformedPath)
} }
@ -519,6 +580,12 @@ impl ProviderClient for ProviderOrchestrator {
.create_lib_node(parent_path, title) .create_lib_node(parent_path, title)
.await; .await;
} }
if youtube_owns(parent_path) {
return self
.youtube_provider()?
.create_lib_node(parent_path, title)
.await;
}
warn!(parent_path, "no provider supports creating nodes here"); warn!(parent_path, "no provider supports creating nodes here");
Err(ProviderError::NotSupported) Err(ProviderError::NotSupported)
} }
@ -555,6 +622,12 @@ impl ProviderClient for ProviderOrchestrator {
.rename_lib_node(path, new_title) .rename_lib_node(path, new_title)
.await; .await;
} }
if youtube_owns(path) {
return self
.youtube_provider()?
.rename_lib_node(path, new_title)
.await;
}
warn!(path, "no provider supports renaming this node"); warn!(path, "no provider supports renaming this node");
Err(ProviderError::NotSupported) Err(ProviderError::NotSupported)
} }
@ -594,6 +667,12 @@ impl ProviderClient for ProviderOrchestrator {
.resolve_tracks_into(path, chunk_tx) .resolve_tracks_into(path, chunk_tx)
.await; .await;
} }
if youtube_owns(path) {
return self
.youtube_provider()?
.resolve_tracks_into(path, chunk_tx)
.await;
}
warn!(path, "no provider owns this path"); warn!(path, "no provider owns this path");
Err(ProviderError::MalformedPath) Err(ProviderError::MalformedPath)
} }
@ -617,6 +696,9 @@ impl ProviderClient for ProviderOrchestrator {
if captures_owns(path) { if captures_owns(path) {
return self.captures_provider()?.delete_lib_node(path).await; return self.captures_provider()?.delete_lib_node(path).await;
} }
if youtube_owns(path) {
return self.youtube_provider()?.delete_lib_node(path).await;
}
warn!(path, "no provider supports deleting this node"); warn!(path, "no provider supports deleting this node");
Err(ProviderError::NotSupported) Err(ProviderError::NotSupported)
} }

View File

@ -14,6 +14,8 @@ let
d2 d2
pkg-config pkg-config
protobuf protobuf
# Extraction engine for the ytdy provider (architecture/youtube-provider.md D1).
yt-dlp
]; ];
in in
{ {

View File

@ -1,5 +1,38 @@
# Implementation summaries # Implementation summaries
## youtube-provider (2026-07-21)
Built per `plan/youtube-provider.md`: a new workspace crate **`ytdy`**
mounts YouTube at `/youtube`, backed by a `yt-dlp` subprocess (declared
in `devenv.nix`). All extraction goes through one `Engine` seam:
argv-only invocations with `--no-warnings`, an optional `--cookies`
flag, a per-call timeout (`kill_on_drop`), a 32 MiB stdout cap, and
typed `EngineError`s — tests drive the whole provider through a fake
shell-script binary, no network.
Search needs no login and mirrors tidal's search exactly: `%` on
`/youtube/search` creates an in-memory term (deduplicated, implicitly
recreated on stale paths, rename re-searches, delete idempotent), whose
node lists the top N (`ytsearchN:`, default 20) results as queueable,
downloadable tracks. With a readable cookies file configured in
`ytdy.toml` ("logged in"), a `playlists` subtree appears
(`feed/playlists` flat listing → playlist nodes with tracks); an
unreadable cookies file degrades to logged-out with a warning, never a
failed init. Streams resolve via `-f bestaudio/best -g`; captures work
end to end (`extension_for` gained `audio/webm → webm`). The
orchestrator wires `/youtube` non-fatally: a failed `--version` probe
disables the provider, nothing else.
Deviations: `Entry` keeps separate `uploader`/`channel` fields with an
`artist()` preference — the planned serde alias rejects real yt-dlp
output as a duplicate field (found by the live probe). The live probe
validated search, stream resolution, and a real download capture
(252 KB webm) through the capture store; the **playlists feed
invocation is live-unvalidated** (no cookies on this machine) — flagged
in `architecture/youtube-provider.md` as the standing risk. 150
workspace tests green (8 new in `ytdy`); every gate in
`quality/youtube-provider.md` checked.
## captures (2026-07-21) ## captures (2026-07-21)
Built per `plan/captures.md`: `W` (shift) on a downloadable library Built per `plan/captures.md`: `W` (shift) on a downloadable library

45
plan/youtube-provider.md Normal file
View File

@ -0,0 +1,45 @@
# Plan: youtube-provider
Ordered tasks; each names its verification (tests in `ytdy/src/lib.rs`
and/or gates in `quality/youtube-provider.md`). The crate skeleton,
stubs, workspace/devenv wiring, and all tests exist; the tests fail on
`todo!()` at plan time.
- [x] **T1 — Engine.** `run` (argv-only, `--no-warnings`, cookie flag,
timeout with `kill_on_drop`, stdout cap, typed errors), `probe`,
`flat_listing`, `video_entry`, `stream_urls`. Verifies:
`engine_failures_are_typed_never_panics`, parts of every other test;
gates "Engine".
- [x] **T2 — Path parsing.** `parse_path` for all `YtPath` shapes;
reject everything else. Verifies:
`foreign_and_malformed_paths_are_rejected`,
`tracks_resolve_streams_and_metadata` (is_track_path).
- [x] **T3 — Init.** Settings parse (defaults on broken TOML like the
other providers), engine construction, `--version` probe, cookies
readability check (degrade to logged out with a warning). Verifies:
`init_probes_the_binary`, `root_lists_playlists_only_when_logged_in`.
- [x] **T4 — Search.** Term registry (snapshot/register),
`search_term_node` (`ytsearchN:`), `create/rename/delete_lib_node`,
`get_lib_node` arms for Root/Search/SearchTerm, central
downloadable rule. Verifies:
`search_terms_are_created_listed_and_searched`,
`search_terms_rename_and_delete`.
- [x] **T5 — Playlists + tracks.** `playlists_node` (feed URL, login
gate), `playlist_node`, `get_urls_for_track` (`-g`),
`get_metadata_for_track` (single-video `-J`). Verifies:
`playlists_list_and_resolve_when_logged_in`,
`tracks_resolve_streams_and_metadata`.
- [x] **T6 — Orchestrator wiring.** `youtube_client: Option<Arc<_>>`,
non-fatal init from `ytdy.toml` (written back), routing arms in every
method, root child; `extension_for` gains `audio/webm`. Verifies:
existing orchestrator patterns by inspection; gates "Orchestrator
wiring"; `extension_prefers_content_type_then_url_then_bin` extended.
- [x] **T7 — Full verification.** Workspace suite green; clippy/fmt/
taplo/markdownlint clean; every gate ticked; no `todo!()`.
- [x] **T8 — Live smoke test.** With the devenv `yt-dlp`: search a term
through the provider, fetch a stream URL for one result, and download
a small capture through the capture store — remove the probe
afterwards. Validate the playlists feed invocation if cookies are
available; otherwise record it as unvalidated in the summary.
- [x] **T9 — Docs.** `plan/summary.md` section incl. deviations;
reconcile `architecture/youtube-provider.md`.

View File

@ -0,0 +1,57 @@
# Quality gates: youtube-provider
Criteria beyond the automatic tests (`ytdy/src/lib.rs`,
`ytdy/src/engine.rs`, plus orchestrator wiring). Each gate is pass/fail
by reading the code.
## Engine (subprocess discipline)
- [x] Every invocation is an argument list (`tokio::process::Command`)
— never a shell string; user input (terms, ids) can contain anything.
- [x] Every call is bounded: per-call timeout (`kill_on_drop` so a timed
out process dies), and captured stdout capped at `MAX_STDOUT_BYTES`.
- [x] All failures are typed `EngineError`s mapped to `ProviderError`
at the trait boundary; no subprocess condition panics.
- [x] Error/log output carries the binary path, exit status, and a
*bounded* stderr summary — never full stderr, never stream URLs,
never cookie file contents (the path alone is loggable).
- [x] The cookies flag is appended to every call when configured;
nothing else about login is stored or invented.
## Provider semantics
- [x] Search terms mirror tidal's: in-memory, deduplicated, implicit
recreation on stale paths, rename re-searches, delete idempotent —
and only `/youtube/search` children are creatable/editable/deletable.
- [x] Path parsing rejects foreign roots and malformed shapes with
`MalformedPath`; track paths are exactly `<node>/<videoid>`.
- [x] Downloadability follows the central rule (queueable or lists
tracks; children mirror `is_queable`) applied in one place, not
per-arm.
- [x] `duration` floats are truncated to whole seconds; missing
title/uploader degrade to empty strings, never errors.
- [x] The playlists subtree exists only when logged in: absent from the
root listing and `MalformedPath` when addressed directly.
## Orchestrator wiring
- [x] `ytdy::Client::init` failure (missing/broken binary) disables the
provider with a warning — the server and every other provider keep
running; `get_lib_root` lists `youtube` only when enabled.
- [x] `/youtube` is routed in every `ProviderClient` method (same
completeness as `/tidal`), including `resolve_tracks_into`,
create/rename/delete.
- [x] `ytdy.toml` is written back after init like the other providers'
configs.
- [x] `extension_for` maps `audio/webm → webm` so captures of YouTube
audio get a sensible extension.
## Hygiene
- [x] `yt-dlp` is declared in `devenv.nix`, never assumed installed.
- [x] New public items are documented; docs state error/edge behavior.
- [x] `clippy -D warnings`, `fmt`, `taplo`, `markdownlint` clean; all
tests green; no `todo!()` left.
- [x] `architecture/youtube-provider.md` reconciled where the
implementation diverged (esp. the playlists-feed invocation after the
live probe).

24
ytdy/Cargo.toml Normal file
View File

@ -0,0 +1,24 @@
[package]
name = "ytdy"
version.workspace = true
edition.workspace = true
[dependencies]
async-trait.workspace = true
crabidy-core.workspace = true
serde.workspace = true
serde_json.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = [
"time",
"sync",
"macros",
"process",
"io-util",
] }
toml.workspace = true
tracing.workspace = true
[dev-dependencies]
tempfile.workspace = true
tokio = { workspace = true, features = ["full"] }

178
ytdy/src/engine.rs Normal file
View File

@ -0,0 +1,178 @@
//! The yt-dlp subprocess seam (see `architecture/youtube-provider.md` D1/D5).
//!
//! Every YouTube interaction is one bounded `yt-dlp` invocation: argument
//! list only (never a shell), `--no-warnings`, an optional `--cookies`
//! file, a per-call timeout, and a cap on captured stdout. JSON output is
//! parsed with `serde_json`; failures are typed [`EngineError`]s — no
//! subprocess condition may panic. Tests point `binary` at a fake script.
use std::path::PathBuf;
use std::time::Duration;
use serde::Deserialize;
/// Captured stdout beyond this many bytes aborts the call — a runaway
/// extractor must not balloon memory.
pub const MAX_STDOUT_BYTES: usize = 32 * 1024 * 1024;
/// Errors from one engine invocation.
///
/// Messages carry the binary name, exit status, and a short stderr
/// summary — never full stderr (it can embed URLs) and never cookie
/// contents.
#[derive(Debug, thiserror::Error)]
pub enum EngineError {
#[error("cannot run yt-dlp: {0}")]
Spawn(#[from] std::io::Error),
#[error("yt-dlp timed out after {0:?}")]
Timeout(Duration),
#[error("yt-dlp failed ({status}): {summary}")]
Failed { status: String, summary: String },
#[error("cannot parse yt-dlp output: {0}")]
Parse(#[from] serde_json::Error),
#[error("yt-dlp output exceeded {MAX_STDOUT_BYTES} bytes")]
Oversized,
}
/// One entry of a `--flat-playlist -J` listing (or a single-video `-J`).
/// Fields we don't use are ignored; absent fields stay `None` — flat
/// extraction fills what it cheaply can.
#[derive(Debug, Clone, Deserialize)]
pub struct Entry {
pub id: String,
#[serde(default)]
pub title: Option<String>,
/// Real yt-dlp output carries `uploader` *and* `channel` (so a serde
/// alias would reject it as a duplicate field); both are kept and
/// [`Self::artist`] picks.
#[serde(default)]
pub uploader: Option<String>,
#[serde(default)]
pub channel: Option<String>,
/// Seconds; yt-dlp emits floats.
#[serde(default)]
pub duration: Option<f64>,
}
impl Entry {
/// The track artist: the uploader, falling back to the channel.
pub fn artist(&self) -> Option<&str> {
self.uploader.as_deref().or(self.channel.as_deref())
}
}
/// A `-J` playlist-shaped result: its `entries`, plus the playlist title.
#[derive(Debug, Deserialize)]
pub struct Listing {
#[serde(default)]
pub title: Option<String>,
#[serde(default)]
pub entries: Vec<Entry>,
}
/// The shared subprocess runner. Cheap to clone-by-reference (the client
/// holds one); all state is configuration.
#[derive(Debug)]
pub struct Engine {
binary: PathBuf,
cookies: Option<PathBuf>,
timeout: Duration,
}
impl Engine {
/// An engine invoking `binary` with `timeout` per call, passing
/// `--cookies <cookies>` on every call when set.
pub fn new(binary: PathBuf, cookies: Option<PathBuf>, timeout: Duration) -> Self {
Self {
binary,
cookies,
timeout,
}
}
/// `yt-dlp --version` — the init probe. The trimmed version string on
/// success.
pub async fn probe(&self) -> Result<String, EngineError> {
let stdout = self.run(&["--version"]).await?;
Ok(String::from_utf8_lossy(&stdout).trim().to_string())
}
/// A flat listing (`-J --flat-playlist`) of `target` — a
/// `ytsearchN:` term, a playlist URL, or the playlists feed.
pub async fn flat_listing(&self, target: &str) -> Result<Listing, EngineError> {
let stdout = self.run(&["-J", "--flat-playlist", target]).await?;
Ok(serde_json::from_slice(&stdout)?)
}
/// Metadata of one video (`-J --no-playlist`).
pub async fn video_entry(&self, video_url: &str) -> Result<Entry, EngineError> {
let stdout = self.run(&["-J", "--no-playlist", video_url]).await?;
Ok(serde_json::from_slice(&stdout)?)
}
/// Stream URL(s) of one video (`-f bestaudio/best -g --no-playlist`),
/// one per line.
pub async fn stream_urls(&self, video_url: &str) -> Result<Vec<String>, EngineError> {
let stdout = self
.run(&["-f", "bestaudio/best", "-g", "--no-playlist", video_url])
.await?;
Ok(String::from_utf8_lossy(&stdout)
.lines()
.map(str::trim)
.filter(|line| !line.is_empty())
.map(str::to_string)
.collect())
}
/// Runs the binary with `args` (plus `--no-warnings` and the cookie
/// flag), enforcing the timeout and stdout cap; returns raw stdout.
///
/// The child is `kill_on_drop`: hitting the timeout kills it instead
/// of leaking a hung extractor.
async fn run(&self, args: &[&str]) -> Result<Vec<u8>, EngineError> {
use std::process::Stdio;
use tokio::io::AsyncReadExt;
let mut cmd = tokio::process::Command::new(&self.binary);
cmd.arg("--no-warnings");
if let Some(cookies) = &self.cookies {
cmd.arg("--cookies").arg(cookies);
}
cmd.args(args)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.kill_on_drop(true);
let mut child = cmd.spawn()?;
// Read stdout incrementally with a hard cap — `wait_with_output`
// alone would buffer an unbounded stream first.
let mut stdout_pipe = child.stdout.take().ok_or_else(|| {
std::io::Error::other("child stdout pipe missing despite Stdio::piped")
})?;
let bounded = async {
let mut stdout = Vec::new();
let mut limited = (&mut stdout_pipe).take(MAX_STDOUT_BYTES as u64 + 1);
limited.read_to_end(&mut stdout).await?;
let rest = child.wait_with_output().await?;
Ok::<_, std::io::Error>((stdout, rest))
};
let (stdout, rest) = tokio::time::timeout(self.timeout, bounded)
.await
.map_err(|_| EngineError::Timeout(self.timeout))??;
if stdout.len() > MAX_STDOUT_BYTES {
return Err(EngineError::Oversized);
}
if !rest.status.success() {
// A bounded summary only: full stderr can embed URLs.
let summary: String = String::from_utf8_lossy(&rest.stderr)
.chars()
.take(200)
.collect();
return Err(EngineError::Failed {
status: rest.status.to_string(),
summary: summary.trim().to_string(),
});
}
Ok(stdout)
}
}

806
ytdy/src/lib.rs Normal file
View File

@ -0,0 +1,806 @@
//! YouTube media provider, backed by a `yt-dlp` subprocess
//! (see `architecture/youtube-provider.md`).
//!
//! Mounted at [`PROVIDER_ROOT`]. Search works without login: creatable
//! search-term nodes exactly like `/tidal/search` (in-memory terms,
//! renamable/deletable, results listed as tracks). With a cookies file
//! configured ("logged in"), the user's playlists appear under
//! `/youtube/playlists`. Every node that serves tracks is downloadable —
//! `W` captures work out of the box.
use std::path::PathBuf;
use std::time::Duration;
use async_trait::async_trait;
use crabidy_core::proto::crabidy::{LibraryNode, LibraryNodeChild, Track};
use crabidy_core::{ProviderClient, ProviderError};
use serde::{Deserialize, Serialize};
use tracing::{debug, warn};
pub mod engine;
use engine::Engine;
/// First path segment owned by this provider.
pub const PROVIDER_ROOT: &str = "/youtube";
/// Default number of search results per term (`ytsearchN:`).
pub const DEFAULT_SEARCH_RESULTS: usize = 20;
/// Default per-subprocess-call timeout in seconds.
pub const DEFAULT_CALL_TIMEOUT_SECS: u64 = 60;
/// Provider settings, persisted as `ytdy.toml` next to the other crabidy
/// config files. The cookies file is user-provided and its **contents are
/// a secret**: only the path may ever be logged.
#[derive(Clone, Debug, Default, Deserialize, Serialize)]
pub struct Settings {
/// The yt-dlp binary; a bare name resolves via PATH. Default
/// `yt-dlp`.
pub binary: Option<PathBuf>,
/// Netscape cookies file for logged-in features (playlists,
/// age-gated streams). Absent = logged out; search still works.
pub cookies: Option<PathBuf>,
/// Results per search term. Default [`DEFAULT_SEARCH_RESULTS`].
pub search_results: Option<usize>,
/// Per-call timeout in seconds. Default
/// [`DEFAULT_CALL_TIMEOUT_SECS`]; tests shrink it.
pub call_timeout_secs: Option<u64>,
}
/// A parsed `/youtube/...` path.
#[derive(Debug, PartialEq, Eq)]
enum YtPath<'a> {
Root,
Search,
/// Percent-encoded search term segment.
SearchTerm(&'a str),
SearchTrack {
term: &'a str,
video: &'a str,
},
Playlists,
Playlist(&'a str),
PlaylistTrack {
playlist: &'a str,
video: &'a str,
},
}
/// Splits a `/youtube/...` path into its recognized shape.
/// Unknown shapes are [`ProviderError::MalformedPath`].
fn parse_path(path: &str) -> Result<YtPath<'_>, ProviderError> {
if path == PROVIDER_ROOT {
return Ok(YtPath::Root);
}
let rest = path
.strip_prefix("/youtube/")
.ok_or(ProviderError::MalformedPath)?;
let segments: Vec<&str> = rest.split('/').collect();
if segments.iter().any(|segment| segment.is_empty()) {
return Err(ProviderError::MalformedPath);
}
match segments.as_slice() {
["search"] => Ok(YtPath::Search),
["search", term] => Ok(YtPath::SearchTerm(term)),
["search", term, video] => Ok(YtPath::SearchTrack { term, video }),
["playlists"] => Ok(YtPath::Playlists),
["playlists", playlist] => Ok(YtPath::Playlist(playlist)),
["playlists", playlist, video] => Ok(YtPath::PlaylistTrack { playlist, video }),
_ => Err(ProviderError::MalformedPath),
}
}
/// Maps an engine failure to the trait-level error, logging the typed
/// cause (paths and statuses only — never URLs or cookie contents).
fn engine_err(context: &str, err: engine::EngineError) -> ProviderError {
warn!(context, "yt-dlp call failed: {err}");
ProviderError::FetchError
}
/// The canonical watch URL for a video id.
fn video_url(video_id: &str) -> String {
format!("https://www.youtube.com/watch?v={video_id}")
}
/// Builds the wire track for one listing entry under `node_path`.
/// Missing metadata degrades to empty fields, never an error; float
/// durations truncate to whole seconds.
fn entry_to_track(entry: &engine::Entry, node_path: &str) -> Track {
Track {
path: crabidy_core::join_path(node_path, &entry.id),
artist: entry.artist().unwrap_or_default().to_string(),
title: entry.title.clone().unwrap_or_default(),
duration: entry.duration.map(|secs| secs.max(0.0) as u32),
album: None,
}
}
/// The YouTube provider client.
#[derive(Debug)]
pub struct Client {
engine: Engine,
settings: Settings,
/// Cookies were configured and readable at init — gates the
/// playlists subtree.
logged_in: bool,
/// Search terms created under `/youtube/search`, in creation order,
/// deduplicated. In-memory only, like tidal's. Never held across
/// awaits.
search_terms: std::sync::RwLock<Vec<String>>,
}
impl Client {
/// The search-term node: top `search_results` results as tracks
/// (`ytsearchN:<term>` flat listing). Queueable and downloadable —
/// results are homogeneous tracks.
async fn search_term_node(
&self,
path: &str,
term: &str,
parent: String,
) -> Result<LibraryNode, ProviderError> {
let count = self
.settings
.search_results
.unwrap_or(DEFAULT_SEARCH_RESULTS);
let listing = self
.engine
.flat_listing(&format!("ytsearch{count}:{term}"))
.await
.map_err(|err| engine_err("search", err))?;
Ok(LibraryNode {
path: path.to_string(),
title: term.to_string(),
parent: Some(parent),
tracks: listing
.entries
.iter()
.map(|entry| entry_to_track(entry, path))
.collect(),
children: Vec::new(),
is_queable: true,
is_creatable: false,
is_downloadable: false,
})
}
/// The playlists listing (`feed/playlists`, cookies required):
/// one queueable, downloadable child per playlist.
async fn playlists_node(
&self,
path: &str,
parent: String,
) -> Result<LibraryNode, ProviderError> {
let listing = self
.engine
.flat_listing("https://www.youtube.com/feed/playlists")
.await
.map_err(|err| engine_err("playlists feed", err))?;
Ok(LibraryNode {
path: path.to_string(),
title: "playlists".to_string(),
parent: Some(parent),
tracks: Vec::new(),
children: listing
.entries
.iter()
.map(|entry| {
LibraryNodeChild::new(
crabidy_core::join_path(path, &entry.id),
entry.title.clone().unwrap_or_else(|| entry.id.clone()),
true,
)
})
.collect(),
is_queable: false,
is_creatable: false,
is_downloadable: false,
})
}
/// One playlist's entries as tracks.
async fn playlist_node(
&self,
path: &str,
playlist_id: &str,
parent: String,
) -> Result<LibraryNode, ProviderError> {
let listing = self
.engine
.flat_listing(&format!(
"https://www.youtube.com/playlist?list={playlist_id}"
))
.await
.map_err(|err| engine_err("playlist", err))?;
Ok(LibraryNode {
path: path.to_string(),
title: listing
.title
.clone()
.unwrap_or_else(|| playlist_id.to_string()),
parent: Some(parent),
tracks: listing
.entries
.iter()
.map(|entry| entry_to_track(entry, path))
.collect(),
children: Vec::new(),
is_queable: true,
is_creatable: false,
is_downloadable: false,
})
}
fn search_terms_snapshot(&self) -> Vec<String> {
self.search_terms
.read()
.map(|terms| terms.clone())
.unwrap_or_default()
}
fn register_search_term(&self, term: &str) {
if let Ok(mut terms) = self.search_terms.write() {
if !terms.iter().any(|existing| existing == term) {
terms.push(term.to_string());
}
}
}
/// Removes a term; `true` when it existed.
fn remove_search_term(&self, term: &str) -> bool {
match self.search_terms.write() {
Ok(mut terms) => {
let before = terms.len();
terms.retain(|existing| existing != term);
terms.len() != before
}
Err(_) => false,
}
}
}
#[async_trait]
impl ProviderClient for Client {
/// Builds the engine from settings and probes `--version`; a missing
/// or broken binary fails init (the orchestrator disables the
/// provider non-fatally). A configured but unreadable cookies file
/// degrades to logged-out with a warning, never an error.
async fn init(raw_toml_settings: &str) -> Result<Self, ProviderError> {
let settings: Settings = toml::from_str(raw_toml_settings).unwrap_or_else(|_| {
warn!("could not parse toml settings, using defaults");
Settings::default()
});
let binary = settings
.binary
.clone()
.unwrap_or_else(|| PathBuf::from("yt-dlp"));
let timeout = Duration::from_secs(
settings
.call_timeout_secs
.unwrap_or(DEFAULT_CALL_TIMEOUT_SECS),
);
// "Logged in" is exactly "the configured cookies file is
// readable"; anything less degrades to logged out, never to a
// failed init. Only the *path* is ever logged.
let logged_in = match &settings.cookies {
Some(path) => match std::fs::metadata(path) {
Ok(meta) if meta.is_file() => true,
_ => {
warn!(
cookies = %path.display(),
"cookies file not readable; running logged out"
);
false
}
},
None => false,
};
let cookies = logged_in.then(|| settings.cookies.clone()).flatten();
let engine = Engine::new(binary, cookies, timeout);
let version = engine.probe().await.map_err(|err| {
warn!("yt-dlp probe failed: {err}");
ProviderError::Config(err.to_string())
})?;
debug!(version, logged_in, "yt-dlp ready");
Ok(Self {
engine,
settings,
logged_in,
search_terms: std::sync::RwLock::new(Vec::new()),
})
}
fn settings(&self) -> String {
toml::to_string_pretty(&self.settings).unwrap_or_default()
}
fn is_track_path(&self, path: &str) -> bool {
matches!(
parse_path(path),
Ok(YtPath::SearchTrack { .. } | YtPath::PlaylistTrack { .. })
)
}
/// `-f bestaudio/best -g` on the track's video id.
async fn get_urls_for_track(&self, track_path: &str) -> Result<Vec<String>, ProviderError> {
let video = match parse_path(track_path)? {
YtPath::SearchTrack { video, .. } | YtPath::PlaylistTrack { video, .. } => video,
_ => return Err(ProviderError::MalformedPath),
};
let urls = self
.engine
.stream_urls(&video_url(video))
.await
.map_err(|err| engine_err("stream urls", err))?;
if urls.is_empty() {
warn!(path = track_path, "yt-dlp returned no stream url");
return Err(ProviderError::FetchError);
}
Ok(urls)
}
/// Single-video `-J` metadata.
async fn get_metadata_for_track(&self, track_path: &str) -> Result<Track, ProviderError> {
let video = match parse_path(track_path)? {
YtPath::SearchTrack { video, .. } | YtPath::PlaylistTrack { video, .. } => video,
_ => return Err(ProviderError::MalformedPath),
};
let entry = self
.engine
.video_entry(&video_url(video))
.await
.map_err(|err| engine_err("video metadata", err))?;
let parent = crabidy_core::parent_path(track_path).unwrap_or(PROVIDER_ROOT);
let mut track = entry_to_track(&entry, parent);
// The entry's id names the video; the caller's path is canonical.
track.path = track_path.to_string();
Ok(track)
}
/// `search` always; `playlists` only when logged in.
fn get_lib_root(&self) -> LibraryNode {
let mut children = vec![LibraryNodeChild {
is_creatable: true,
..LibraryNodeChild::new(
format!("{PROVIDER_ROOT}/search"),
"search".to_string(),
false,
)
}];
if self.logged_in {
children.push(LibraryNodeChild::new(
format!("{PROVIDER_ROOT}/playlists"),
"playlists".to_string(),
false,
));
}
LibraryNode {
path: PROVIDER_ROOT.to_string(),
title: "youtube".to_string(),
parent: Some(crabidy_core::ROOT_PATH.to_string()),
tracks: Vec::new(),
children,
is_queable: false,
is_creatable: false,
is_downloadable: false,
}
}
async fn get_lib_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
let parent = crabidy_core::parent_path(path)
.unwrap_or(crabidy_core::ROOT_PATH)
.to_string();
let node = match parse_path(path)? {
YtPath::Root => self.get_lib_root(),
YtPath::Search => LibraryNode {
path: path.to_string(),
title: "search".to_string(),
parent: Some(parent),
tracks: Vec::new(),
children: self
.search_terms_snapshot()
.iter()
.map(|term| {
// Term nodes are the modifiable nodes: renamable
// (`e`) and deletable (`d`), like tidal's.
LibraryNodeChild {
is_editable: true,
is_deletable: true,
..LibraryNodeChild::new(
crabidy_core::join_path(path, &crabidy_core::encode_segment(term)),
term.clone(),
true,
)
}
})
.collect(),
is_queable: false,
is_creatable: true,
is_downloadable: false,
},
YtPath::SearchTerm(encoded) => {
let term = crabidy_core::decode_segment(encoded);
// Unknown terms (stale client cache, server restart) are
// recreated implicitly instead of erroring.
self.register_search_term(&term);
self.search_term_node(path, &term, parent).await?
}
YtPath::Playlists => {
if !self.logged_in {
warn!(path, "playlists need a configured cookies file");
return Err(ProviderError::MalformedPath);
}
self.playlists_node(path, parent).await?
}
YtPath::Playlist(playlist_id) => {
if !self.logged_in {
warn!(path, "playlists need a configured cookies file");
return Err(ProviderError::MalformedPath);
}
self.playlist_node(path, playlist_id, parent).await?
}
YtPath::SearchTrack { .. } | YtPath::PlaylistTrack { .. } => {
warn!(path, "get_lib_node called with a track path");
return Err(ProviderError::MalformedPath);
}
};
// The central download blessing (architecture/youtube-provider.md
// D4, same rule as tidal): every node serving playable content
// allows `W`; children mirror their queueability.
let mut node = node;
node.is_downloadable = node.is_queable || !node.tracks.is_empty();
for child in &mut node.children {
child.is_downloadable = child.is_queable;
}
Ok(node)
}
/// Only `/youtube/search` is creatable: registers the term and
/// returns its node (implicit recreation on stale paths, like
/// tidal).
async fn create_lib_node(
&self,
parent_path: &str,
title: &str,
) -> Result<LibraryNode, ProviderError> {
let term = title.trim();
if term.is_empty() {
return Err(ProviderError::InvalidInput);
}
if parse_path(parent_path)? != YtPath::Search {
warn!(parent_path, "node creation not supported here");
return Err(ProviderError::NotSupported);
}
self.register_search_term(term);
let term_path = crabidy_core::join_path(parent_path, &crabidy_core::encode_segment(term));
self.get_lib_node(&term_path).await
}
/// Renaming a search term re-runs the search under the new term.
async fn rename_lib_node(
&self,
path: &str,
new_title: &str,
) -> Result<LibraryNode, ProviderError> {
let YtPath::SearchTerm(encoded) = parse_path(path)? else {
warn!(path, "only search terms are renamable");
return Err(ProviderError::NotSupported);
};
let new_term = new_title.trim();
if new_term.is_empty() {
return Err(ProviderError::InvalidInput);
}
let old_term = crabidy_core::decode_segment(encoded);
// Replace in place; renaming onto an existing term merges (the
// duplicate disappears), like tidal's terms.
self.remove_search_term(&old_term);
self.register_search_term(new_term);
let new_path =
crabidy_core::join_path("/youtube/search", &crabidy_core::encode_segment(new_term));
self.get_lib_node(&new_path).await
}
/// Deleting a search term is idempotent and returns the refreshed
/// search node.
async fn delete_lib_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
let YtPath::SearchTerm(encoded) = parse_path(path)? else {
warn!(path, "only search terms are deletable");
return Err(ProviderError::NotSupported);
};
let term = crabidy_core::decode_segment(encoded);
self.remove_search_term(&term);
self.get_lib_node("/youtube/search").await
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::fs;
use std::os::unix::fs::PermissionsExt;
use std::path::Path;
use tempfile::TempDir;
/// A fake yt-dlp: a shell script dispatching on its argv. Every test
/// engine call goes through it — no network, no real binary.
fn fake_binary(dir: &Path, body: &str) -> PathBuf {
let path = dir.join("fake-yt-dlp");
fs::write(&path, format!("#!/bin/sh\n{body}\n")).expect("write fake binary");
let mut perms = fs::metadata(&path).expect("metadata").permissions();
perms.set_mode(0o755);
fs::set_permissions(&path, perms).expect("chmod");
path
}
/// The standard fake: search terms `lofi` (2 results), one playlist
/// feed, one playlist, and one resolvable video.
const FAKE: &str = r#"
case "$*" in
*--version*) echo "2026.06.09"; exit 0 ;;
*ytsearch2:lofi\ beats*) printf '%s' '{"entries":[{"id":"vidA","title":"Beats","uploader":"Chan"}]}'; exit 0 ;;
*ytsearch2:lofi*) printf '%s' '{"entries":[{"id":"vid1","title":"Track One","uploader":"Chan","duration":63.4},{"id":"vid2","title":"Track Two","channel":"Chan Two"}]}'; exit 0 ;;
*-g*watch?v=vid1*|*watch?v=vid1*-g*) printf '%s\n%s\n' "https://example.test/a.webm" "https://example.test/b.webm"; exit 0 ;;
*watch?v=vid1*) printf '%s' '{"id":"vid1","title":"Track One","uploader":"Chan","duration":63.9}'; exit 0 ;;
*feed/playlists*) printf '%s' '{"entries":[{"id":"PL1","title":"Road Mix"}]}'; exit 0 ;;
*list=PL1*) printf '%s' '{"title":"Road Mix","entries":[{"id":"vid9","title":"Nine","uploader":"Chan","duration":10}]}'; exit 0 ;;
*) echo "unmatched: $*" >&2; exit 1 ;;
esac"#;
async fn client_with(dir: &Path, body: &str, cookies: Option<&Path>) -> Client {
let binary = fake_binary(dir, body);
let settings = Settings {
binary: Some(binary),
cookies: cookies.map(Path::to_path_buf),
search_results: Some(2),
// Generous: under full-workspace parallel test load, process
// spawn latency has flaked a 5 s budget.
call_timeout_secs: Some(30),
};
let toml = toml::to_string(&settings).expect("settings toml");
Client::init(&toml).await.expect("init")
}
async fn client(dir: &Path) -> Client {
client_with(dir, FAKE, None).await
}
#[tokio::test]
async fn init_probes_the_binary() {
let dir = TempDir::new().expect("tempdir");
// A working probe succeeds…
let _ = client(dir.path()).await;
// …a missing binary fails init (the orchestrator treats that as
// "provider disabled", never as a server error).
let settings = Settings {
binary: Some(dir.path().join("no-such-binary")),
..Settings::default()
};
let toml = toml::to_string(&settings).expect("settings toml");
assert!(Client::init(&toml).await.is_err());
// …and so does one that exits non-zero on --version.
let broken = TempDir::new().expect("tempdir");
fake_binary(broken.path(), "exit 3");
let settings = Settings {
binary: Some(broken.path().join("fake-yt-dlp")),
..Settings::default()
};
let toml = toml::to_string(&settings).expect("settings toml");
assert!(Client::init(&toml).await.is_err());
}
#[tokio::test]
async fn root_lists_playlists_only_when_logged_in() {
let dir = TempDir::new().expect("tempdir");
let anon = client(dir.path()).await;
let root = anon.get_lib_root();
let titles: Vec<&str> = root.children.iter().map(|c| c.title.as_str()).collect();
assert_eq!(titles, vec!["search"]);
assert!(root.children[0].is_creatable, "search is creatable");
let cookies = dir.path().join("cookies.txt");
fs::write(&cookies, "# Netscape HTTP Cookie File\n").expect("cookies");
let dir2 = TempDir::new().expect("tempdir");
let logged_in = client_with(dir2.path(), FAKE, Some(&cookies)).await;
let root = logged_in.get_lib_root();
let titles: Vec<&str> = root.children.iter().map(|c| c.title.as_str()).collect();
assert_eq!(titles, vec!["search", "playlists"]);
// A configured but unreadable cookies file degrades to logged
// out instead of failing init.
let dir3 = TempDir::new().expect("tempdir");
let missing = dir3.path().join("gone.txt");
let degraded = client_with(dir3.path(), FAKE, Some(&missing)).await;
assert_eq!(degraded.get_lib_root().children.len(), 1);
}
#[tokio::test]
async fn search_terms_are_created_listed_and_searched() {
let dir = TempDir::new().expect("tempdir");
let client = client(dir.path()).await;
let node = client
.create_lib_node("/youtube/search", "lofi")
.await
.expect("create term");
assert_eq!(node.path, "/youtube/search/lofi");
assert!(node.is_queable, "pure track results are queueable");
assert!(node.is_downloadable, "search results are downloadable");
assert_eq!(node.tracks.len(), 2);
let one = &node.tracks[0];
assert_eq!(one.path, "/youtube/search/lofi/vid1");
assert_eq!(one.title, "Track One");
assert_eq!(one.artist, "Chan");
assert_eq!(one.duration, Some(63));
// `channel` is an accepted alias for the artist.
assert_eq!(node.tracks[1].artist, "Chan Two");
// The search node lists the term as an editable/deletable child.
let search = client
.get_lib_node("/youtube/search")
.await
.expect("search");
assert!(search.is_creatable);
assert_eq!(search.children.len(), 1);
let child = &search.children[0];
assert_eq!(child.title, "lofi");
assert!(child.is_editable && child.is_deletable);
// Encoded terms round trip; unknown terms recreate implicitly.
let node = client
.get_lib_node("/youtube/search/lofi%20beats")
.await
.expect("implicit term");
assert_eq!(node.tracks.len(), 1);
assert_eq!(node.tracks[0].path, "/youtube/search/lofi%20beats/vidA");
assert_eq!(client.search_terms_snapshot().len(), 2);
}
#[tokio::test]
async fn search_terms_rename_and_delete() {
let dir = TempDir::new().expect("tempdir");
let client = client(dir.path()).await;
let _ = client
.create_lib_node("/youtube/search", "lofi beats")
.await
.expect("create term");
let renamed = client
.rename_lib_node("/youtube/search/lofi%20beats", "lofi")
.await
.expect("rename re-searches");
assert_eq!(renamed.path, "/youtube/search/lofi");
assert_eq!(renamed.tracks.len(), 2);
assert_eq!(client.search_terms_snapshot(), vec!["lofi".to_string()]);
let search = client
.delete_lib_node("/youtube/search/lofi")
.await
.expect("delete");
assert!(search.children.is_empty());
// Idempotent: deleting again still returns the search node.
let again = client
.delete_lib_node("/youtube/search/lofi")
.await
.expect("idempotent delete");
assert!(again.children.is_empty());
// Only /youtube/search children are mutable.
assert!(client.rename_lib_node("/youtube", "nope").await.is_err());
assert!(client.delete_lib_node("/youtube/playlists").await.is_err());
}
#[tokio::test]
async fn playlists_list_and_resolve_when_logged_in() {
let dir = TempDir::new().expect("tempdir");
let cookies = dir.path().join("cookies.txt");
fs::write(&cookies, "# cookies\n").expect("cookies");
let client = client_with(dir.path(), FAKE, Some(&cookies)).await;
let playlists = client
.get_lib_node("/youtube/playlists")
.await
.expect("playlists feed");
assert_eq!(playlists.children.len(), 1);
let pl = &playlists.children[0];
assert_eq!(pl.path, "/youtube/playlists/PL1");
assert_eq!(pl.title, "Road Mix");
assert!(pl.is_queable && pl.is_downloadable);
let node = client.get_lib_node(&pl.path).await.expect("playlist");
assert_eq!(node.title, "Road Mix");
assert!(node.is_queable && node.is_downloadable);
assert_eq!(node.tracks.len(), 1);
assert_eq!(node.tracks[0].path, "/youtube/playlists/PL1/vid9");
// Logged out, the playlists subtree is a malformed path.
let dir2 = TempDir::new().expect("tempdir");
let anon = client_with(dir2.path(), FAKE, None).await;
assert!(anon.get_lib_node("/youtube/playlists").await.is_err());
}
#[tokio::test]
async fn tracks_resolve_streams_and_metadata() {
let dir = TempDir::new().expect("tempdir");
let client = client(dir.path()).await;
assert!(client.is_track_path("/youtube/search/lofi/vid1"));
assert!(client.is_track_path("/youtube/playlists/PL1/vid9"));
assert!(!client.is_track_path("/youtube/search/lofi"));
let urls = client
.get_urls_for_track("/youtube/search/lofi/vid1")
.await
.expect("stream urls");
assert_eq!(
urls,
vec![
"https://example.test/a.webm".to_string(),
"https://example.test/b.webm".into()
]
);
let track = client
.get_metadata_for_track("/youtube/playlists/PL1/vid1")
.await
.expect("metadata");
assert_eq!(track.title, "Track One");
assert_eq!(track.artist, "Chan");
assert_eq!(track.duration, Some(63));
assert_eq!(track.path, "/youtube/playlists/PL1/vid1");
}
#[tokio::test]
async fn engine_failures_are_typed_never_panics() {
let dir = TempDir::new().expect("tempdir");
// Non-zero exit on everything but the probe.
let body = r#"
case "$*" in
*--version*) echo 1.0; exit 0 ;;
*) echo boom >&2; exit 1 ;;
esac"#;
let client = client_with(dir.path(), body, None).await;
assert!(client.get_lib_node("/youtube/search/lofi").await.is_err());
assert!(client
.get_urls_for_track("/youtube/search/lofi/vid1")
.await
.is_err());
// Malformed JSON is a typed error, not a panic.
let dir2 = TempDir::new().expect("tempdir");
let body = r#"
case "$*" in
*--version*) echo 1.0; exit 0 ;;
*) printf 'not json'; exit 0 ;;
esac"#;
let client = client_with(dir2.path(), body, None).await;
assert!(client.get_lib_node("/youtube/search/lofi").await.is_err());
// A hung binary hits the per-call timeout.
let dir3 = TempDir::new().expect("tempdir");
let body = r#"
case "$*" in
*--version*) echo 1.0; exit 0 ;;
*) sleep 30 ;;
esac"#;
let binary = fake_binary(dir3.path(), body);
let settings = Settings {
binary: Some(binary),
call_timeout_secs: Some(1),
search_results: Some(2),
..Settings::default()
};
let toml = toml::to_string(&settings).expect("settings toml");
let client = Client::init(&toml).await.expect("init");
let started = std::time::Instant::now();
assert!(client.get_lib_node("/youtube/search/lofi").await.is_err());
assert!(
started.elapsed() < Duration::from_secs(10),
"timed out late"
);
}
#[tokio::test]
async fn foreign_and_malformed_paths_are_rejected() {
let dir = TempDir::new().expect("tempdir");
let client = client(dir.path()).await;
for path in ["/tidal/artists", "/youtube/nope", "/youtube/search/a/b/c"] {
assert!(client.get_lib_node(path).await.is_err(), "{path}");
}
assert!(client.create_lib_node("/youtube", "term").await.is_err());
}
}