Add a YouTube provider backed by yt-dlp
The new ytdy crate mounts /youtube: login-free search terms exactly like tidal's (creatable, renamable, deletable, results as queueable downloadable tracks) and, when a cookies file is configured, the user's playlists. All extraction runs through one bounded subprocess seam (argv-only, per-call timeout, stdout cap, typed errors) so tests drive the provider with a fake script. yt-dlp is declared in devenv; a failed binary probe disables the provider, never the server. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
7f0e900c56
commit
c6e9f71cf4
|
|
@ -760,6 +760,7 @@ dependencies = [
|
||||||
"tracing",
|
"tracing",
|
||||||
"tracing-appender",
|
"tracing-appender",
|
||||||
"tracing-subscriber",
|
"tracing-subscriber",
|
||||||
|
"ytdy",
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
|
|
@ -5133,6 +5134,21 @@ dependencies = [
|
||||||
"synstructure",
|
"synstructure",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "ytdy"
|
||||||
|
version = "0.1.0"
|
||||||
|
dependencies = [
|
||||||
|
"async-trait",
|
||||||
|
"crabidy-core",
|
||||||
|
"serde",
|
||||||
|
"serde_json",
|
||||||
|
"tempfile",
|
||||||
|
"thiserror 2.0.19",
|
||||||
|
"tokio",
|
||||||
|
"toml",
|
||||||
|
"tracing",
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "zbus"
|
name = "zbus"
|
||||||
version = "5.18.0"
|
version = "5.18.0"
|
||||||
|
|
|
||||||
|
|
@ -7,6 +7,7 @@ members = [
|
||||||
"crabidy-server",
|
"crabidy-server",
|
||||||
"fsdy",
|
"fsdy",
|
||||||
"tidaldy",
|
"tidaldy",
|
||||||
|
"ytdy",
|
||||||
]
|
]
|
||||||
|
|
||||||
[workspace.package]
|
[workspace.package]
|
||||||
|
|
@ -68,3 +69,4 @@ audio-player = { path = "audio-player" }
|
||||||
crabidy-core = { path = "crabidy-core" }
|
crabidy-core = { path = "crabidy-core" }
|
||||||
fsdy = { path = "fsdy" }
|
fsdy = { path = "fsdy" }
|
||||||
tidaldy = { path = "tidaldy" }
|
tidaldy = { path = "tidaldy" }
|
||||||
|
ytdy = { path = "ytdy" }
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,186 @@
|
||||||
|
# YouTube provider (ytdy)
|
||||||
|
|
||||||
|
## Context and problem statement
|
||||||
|
|
||||||
|
A new library provider for YouTube, mounted at `/youtube`:
|
||||||
|
|
||||||
|
- **Search** works without any login — creatable search-term nodes like
|
||||||
|
`/tidal/search` (`%` creates a term, results list as tracks).
|
||||||
|
- **Login is optional.** When configured, the user's **playlists** appear
|
||||||
|
as an extra subtree; without it, the provider still works (search
|
||||||
|
only).
|
||||||
|
- **Captures** (`W`, download) must work on YouTube nodes — search
|
||||||
|
results and playlists are downloadable.
|
||||||
|
|
||||||
|
## Assumptions (confirmed)
|
||||||
|
|
||||||
|
- The captures machinery is provider-agnostic: anything that answers
|
||||||
|
`get_lib_node`/`get_urls_for_track` and raises `is_downloadable` gets
|
||||||
|
`w`/`W` for free — no new wire or TUI work at all.
|
||||||
|
- The TUI's creatable/editable/deletable node flows (`%`/`e`/`d`) are
|
||||||
|
generic; mirroring tidal's search-term semantics costs no TUI change.
|
||||||
|
- `pkgs.yt-dlp` (2026.06.09) exists in nixpkgs; the devenv already pins
|
||||||
|
the toolchain, so the engine binary is declared, not assumed.
|
||||||
|
- The audio player streams plain https URLs; googlevideo stream URLs
|
||||||
|
(from `bestaudio`) are plain https and also downloadable with the
|
||||||
|
captures reqwest client.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
### D1 — Extraction engine: `yt-dlp` subprocess
|
||||||
|
|
||||||
|
Options considered:
|
||||||
|
|
||||||
|
- *(a)* **Pure-Rust extractor crates** (`rustypipe`, `rusty_ytdl`): no
|
||||||
|
external binary, but they chase YouTube's extraction changes with
|
||||||
|
small maintainer teams, and logged-in user playlists are weakly or not
|
||||||
|
supported.
|
||||||
|
- *(b)* **Invidious/Piped instances**: no extraction code at all, but a
|
||||||
|
hard runtime dependency on third-party servers of unpredictable
|
||||||
|
availability — worse than a local binary for a self-hosted player.
|
||||||
|
- *(c)* **`yt-dlp` as a subprocess** with `-J` JSON output: the de-facto
|
||||||
|
standard extractor, fastest to track YouTube changes, supports search
|
||||||
|
(`ytsearchN:`), playlists, cookies-based login, and direct stream
|
||||||
|
URLs. Cost: a non-Rust runtime dependency and subprocess plumbing.
|
||||||
|
|
||||||
|
**Decision: (c).** The binary is declared in `devenv.nix` (dev) and is a
|
||||||
|
documented runtime requirement (deploys). All calls go through one
|
||||||
|
`Engine` seam (`tokio::process::Command`, `kill_on_drop`, per-call
|
||||||
|
timeout, bounded stdout, `serde_json` parsing) so tests fake the binary
|
||||||
|
with a script and a future pure-Rust engine stays swappable.
|
||||||
|
|
||||||
|
### D2 — Crate `ytdy`, mounted at `/youtube`, non-fatal init
|
||||||
|
|
||||||
|
New workspace crate `ytdy` implementing `ProviderClient`, following
|
||||||
|
`tidaldy`'s shape. `init` probes `<binary> --version` (with timeout);
|
||||||
|
a missing or broken binary disables the provider with a warning — the
|
||||||
|
server and every other provider keep running. Settings (`ytdy.toml`):
|
||||||
|
|
||||||
|
```toml
|
||||||
|
binary = "yt-dlp" # optional; PATH lookup by default
|
||||||
|
cookies = "/path/cookies.txt" # optional; presence = "logged in"
|
||||||
|
search_results = 20 # ytsearchN cap
|
||||||
|
```
|
||||||
|
|
||||||
|
The orchestrator gains `youtube_client: Option<Arc<ytdy::Client>>` and
|
||||||
|
`/youtube` routing arms (same completeness as the other providers);
|
||||||
|
`get_lib_root` lists `youtube` only when the probe succeeded.
|
||||||
|
|
||||||
|
### D3 — Tree shape
|
||||||
|
|
||||||
|
- `/youtube` — children: `search` (always, `is_creatable`), `playlists`
|
||||||
|
(only when cookies are configured).
|
||||||
|
- `/youtube/search/<term>` — created via `%` like tidal search; terms
|
||||||
|
live in memory (`RwLock<Vec<String>>`, dedup, recreated implicitly on
|
||||||
|
stale paths), renamable and deletable. The node lists the top
|
||||||
|
`search_results` results as **tracks** (`ytsearchN:<term>`,
|
||||||
|
`--flat-playlist`); it is queueable (results are homogeneous tracks,
|
||||||
|
unlike tidal's mixed search).
|
||||||
|
- `/youtube/playlists` — the user's playlists as children
|
||||||
|
(`https://www.youtube.com/feed/playlists` with cookies, flat).
|
||||||
|
- `/youtube/playlists/<pid>` — playlist entries as tracks.
|
||||||
|
- Track paths: `<node>/<videoid>`; metadata from the flat entries
|
||||||
|
(title, uploader as artist, duration). `get_metadata_for_track` runs
|
||||||
|
a single-video `-J` when called directly.
|
||||||
|
|
||||||
|
### D4 — Streams, downloads, login
|
||||||
|
|
||||||
|
- `get_urls_for_track`: `yt-dlp -f bestaudio/best -g --no-playlist
|
||||||
|
<video>` → the stream URL(s). Cookies are passed to **every** call
|
||||||
|
when configured (search and streams benefit too, e.g. age-gated
|
||||||
|
videos).
|
||||||
|
- Downloadability mirrors tidal's central rule: a node is downloadable
|
||||||
|
when it is queueable or lists tracks; children mirror `is_queable`.
|
||||||
|
`extension_for` gains `audio/webm → webm` (bestaudio is usually opus
|
||||||
|
in webm; googlevideo URLs carry no path extension).
|
||||||
|
- "Logged in" is exactly "a cookies file is configured and readable at
|
||||||
|
init" — no OAuth flow of our own, no credential storage beyond the
|
||||||
|
user-provided file. The cookies path is config; its **contents are a
|
||||||
|
secret** and must never appear in logs or error reports.
|
||||||
|
|
||||||
|
### D5 — Subprocess discipline
|
||||||
|
|
||||||
|
- One shared `Engine` with the binary path; every invocation:
|
||||||
|
`--no-warnings -J` (or `-g`), argument-list only (never shell),
|
||||||
|
per-call timeout (default 60 s, configurable for tests), stdout
|
||||||
|
capped, stderr summarized into warnings (never echoed wholesale — it
|
||||||
|
can contain URLs).
|
||||||
|
- Errors are typed (`EngineError`: spawn/timeout/exit-status/parse) and
|
||||||
|
map to `ProviderError::FetchError`/`MalformedPath` at the trait
|
||||||
|
boundary; no panic on any subprocess condition.
|
||||||
|
- Calls run sequentially per request (no engine-level queue); the
|
||||||
|
orchestrator already spawns long walks.
|
||||||
|
|
||||||
|
### D6 — Out of scope (explicitly)
|
||||||
|
|
||||||
|
- OAuth/device login flows, cookie refresh, or storing credentials.
|
||||||
|
- Uploads, likes, subscriptions feeds, comments; YouTube Music.
|
||||||
|
- Caching search or playlist results across calls (each listing is a
|
||||||
|
fresh subprocess call, like tidal's fresh HTTP calls).
|
||||||
|
- SponsorBlock, chapters, DRM-protected content (skipped as unplayable).
|
||||||
|
|
||||||
|
## Structure
|
||||||
|
|
||||||
|
```d2
|
||||||
|
direction: right
|
||||||
|
|
||||||
|
server: crabidy-server {
|
||||||
|
orch: ProviderOrchestrator
|
||||||
|
}
|
||||||
|
|
||||||
|
ytdy: "ytdy (crate)" {
|
||||||
|
client: "Client\n(ProviderClient)"
|
||||||
|
terms: "search terms\n(in-memory, like tidal)"
|
||||||
|
engine: "Engine\n(subprocess seam)"
|
||||||
|
client -> terms
|
||||||
|
client -> engine
|
||||||
|
}
|
||||||
|
|
||||||
|
bin: "yt-dlp binary\n(devenv / PATH)" {
|
||||||
|
shape: hexagon
|
||||||
|
}
|
||||||
|
|
||||||
|
yt: "YouTube" { shape: cloud }
|
||||||
|
cookies: "cookies.txt\n(optional, user-provided)" { shape: page }
|
||||||
|
|
||||||
|
server.orch -> ytdy.client: "/youtube/..."
|
||||||
|
ytdy.engine -> bin: "argv + -J, timeout,\nbounded stdout"
|
||||||
|
bin -> yt
|
||||||
|
cookies -> ytdy.engine: "--cookies (when set)"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key flow: search, then W
|
||||||
|
|
||||||
|
```d2
|
||||||
|
shape: sequence_diagram
|
||||||
|
tui: TUI
|
||||||
|
orch: Orchestrator
|
||||||
|
y: ytdy
|
||||||
|
e: yt-dlp
|
||||||
|
|
||||||
|
tui -> orch: "% on /youtube/search: 'lofi'"
|
||||||
|
orch -> y: "create_lib_node"
|
||||||
|
y -> e: "ytsearch20:lofi -J --flat-playlist"
|
||||||
|
e -> y: "entries (id, title, uploader, duration)"
|
||||||
|
y -> tui: "term node: 20 tracks, downloadable"
|
||||||
|
tui -> orch: "W → CaptureLibraryNode(download)"
|
||||||
|
orch -> y: "per track: get_urls_for_track"
|
||||||
|
y -> e: "-f bestaudio -g <id>"
|
||||||
|
e -> y: "googlevideo url"
|
||||||
|
orch -> orch: "captures store: stream to\n/captures/<name>/ (webm + toml)"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Risks and open questions
|
||||||
|
|
||||||
|
- **`feed/playlists` extraction**: the exact yt-dlp invocation for "my
|
||||||
|
playlists" remains **live-unvalidated** (no cookies were available
|
||||||
|
during development; search, streams, and download captures were
|
||||||
|
validated live). Fallback if it turns out wrong: adjust the feed URL
|
||||||
|
in `playlists_node`, or bookmark playlist URLs as search terms.
|
||||||
|
- **yt-dlp breakage/drift**: extraction can break until the devenv pin
|
||||||
|
is bumped; errors stay typed and non-fatal.
|
||||||
|
- **Stream URL lifetime**: googlevideo URLs expire (~6 h); captures
|
||||||
|
download immediately after resolving, playback resolves on play —
|
||||||
|
both within the window.
|
||||||
|
- **Terms of service**: downloads are personal-use copies of streams the
|
||||||
|
client can already play, same stance as tidal captures.
|
||||||
|
|
@ -28,6 +28,7 @@ tonic.workspace = true
|
||||||
tracing.workspace = true
|
tracing.workspace = true
|
||||||
tracing-appender.workspace = true
|
tracing-appender.workspace = true
|
||||||
tracing-subscriber.workspace = true
|
tracing-subscriber.workspace = true
|
||||||
|
ytdy.workspace = true
|
||||||
|
|
||||||
[dev-dependencies]
|
[dev-dependencies]
|
||||||
tempfile.workspace = true
|
tempfile.workspace = true
|
||||||
|
|
|
||||||
|
|
@ -337,6 +337,9 @@ fn extension_for(content_type: Option<&str>, url: &str) -> String {
|
||||||
"audio/mpeg" | "audio/mp3" => Some("mp3"),
|
"audio/mpeg" | "audio/mp3" => Some("mp3"),
|
||||||
"audio/ogg" => Some("ogg"),
|
"audio/ogg" => Some("ogg"),
|
||||||
"audio/wav" | "audio/x-wav" => Some("wav"),
|
"audio/wav" | "audio/x-wav" => Some("wav"),
|
||||||
|
// YouTube bestaudio is usually opus in webm; googlevideo
|
||||||
|
// URLs carry no path extension to fall back on.
|
||||||
|
"audio/webm" | "video/webm" => Some("webm"),
|
||||||
_ => None,
|
_ => None,
|
||||||
});
|
});
|
||||||
match mapped.or_else(|| url_extension(url)) {
|
match mapped.or_else(|| url_extension(url)) {
|
||||||
|
|
@ -379,6 +382,7 @@ mod tests {
|
||||||
("audio/mpeg", "mp3"),
|
("audio/mpeg", "mp3"),
|
||||||
("audio/ogg", "ogg"),
|
("audio/ogg", "ogg"),
|
||||||
("audio/wav", "wav"),
|
("audio/wav", "wav"),
|
||||||
|
("audio/webm", "webm"),
|
||||||
] {
|
] {
|
||||||
assert_eq!(extension_for(Some(ct), "https://x.test/s"), ext);
|
assert_eq!(extension_for(Some(ct), "https://x.test/s"), ext);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -36,6 +36,9 @@ pub struct ProviderOrchestrator {
|
||||||
/// The download-capture writer; `None` disables download captures
|
/// The download-capture writer; `None` disables download captures
|
||||||
/// (and the `/captures` mount goes with it).
|
/// (and the `/captures` mount goes with it).
|
||||||
capture_store: Option<Arc<CaptureStore>>,
|
capture_store: Option<Arc<CaptureStore>>,
|
||||||
|
/// The YouTube provider (yt-dlp backed); `None` when the binary
|
||||||
|
/// probe failed at init (architecture/youtube-provider.md D2).
|
||||||
|
youtube_client: Option<Arc<ytdy::Client>>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether a path belongs to the filesystem provider.
|
/// Whether a path belongs to the filesystem provider.
|
||||||
|
|
@ -59,6 +62,11 @@ fn captures_owns(path: &str) -> bool {
|
||||||
path == CAPTURES_PROVIDER_ROOT || path.starts_with("/captures/")
|
path == CAPTURES_PROVIDER_ROOT || path.starts_with("/captures/")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Whether a path belongs to the YouTube provider.
|
||||||
|
fn youtube_owns(path: &str) -> bool {
|
||||||
|
path == ytdy::PROVIDER_ROOT || path.starts_with("/youtube/")
|
||||||
|
}
|
||||||
|
|
||||||
impl ProviderOrchestrator {
|
impl ProviderOrchestrator {
|
||||||
/// The fs client, or `MalformedPath` (with a warning) when the
|
/// The fs client, or `MalformedPath` (with a warning) when the
|
||||||
/// provider is disabled — a `/fs` path then has no owner.
|
/// provider is disabled — a `/fs` path then has no owner.
|
||||||
|
|
@ -95,6 +103,15 @@ impl ProviderOrchestrator {
|
||||||
ProviderError::MalformedPath
|
ProviderError::MalformedPath
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The YouTube client, or `MalformedPath` (with a warning) when the
|
||||||
|
/// provider is disabled — a `/youtube` path then has no owner.
|
||||||
|
fn youtube_provider(&self) -> Result<&ytdy::Client, ProviderError> {
|
||||||
|
self.youtube_client.as_deref().ok_or_else(|| {
|
||||||
|
warn!("youtube provider is disabled");
|
||||||
|
ProviderError::MalformedPath
|
||||||
|
})
|
||||||
|
}
|
||||||
pub fn run(self) {
|
pub fn run(self) {
|
||||||
tokio::spawn(async move {
|
tokio::spawn(async move {
|
||||||
// Behind an Arc so long-running resolves can be spawned onto
|
// Behind an Arc so long-running resolves can be spawned onto
|
||||||
|
|
@ -318,6 +335,23 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
// YouTube: non-fatal like the local providers — a missing or
|
||||||
|
// broken yt-dlp binary only costs the `/youtube` subtree.
|
||||||
|
let yt_config_file = config_dir.join("ytdy.toml");
|
||||||
|
debug!(config_file = %yt_config_file.display(), "loading youtube config");
|
||||||
|
let raw_yt_settings = fs::read_to_string(&yt_config_file).unwrap_or_default();
|
||||||
|
let youtube_client = match ytdy::Client::init(&raw_yt_settings).await {
|
||||||
|
Ok(client) => {
|
||||||
|
if let Err(err) = tokio::fs::write(&yt_config_file, client.settings()).await {
|
||||||
|
error!("failed to write ytdy config file: {err}");
|
||||||
|
}
|
||||||
|
Some(Arc::new(client))
|
||||||
|
}
|
||||||
|
Err(err) => {
|
||||||
|
warn!("youtube provider disabled: {err}");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
};
|
||||||
let (provider_tx, provider_rx) = flume::bounded(100);
|
let (provider_tx, provider_rx) = flume::bounded(100);
|
||||||
Ok(Self {
|
Ok(Self {
|
||||||
provider_rx,
|
provider_rx,
|
||||||
|
|
@ -329,6 +363,7 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
bookmark_store,
|
bookmark_store,
|
||||||
captures_client,
|
captures_client,
|
||||||
capture_store,
|
capture_store,
|
||||||
|
youtube_client,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -365,6 +400,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.as_ref()
|
.as_ref()
|
||||||
.is_some_and(|captures| captures.is_track_path(path));
|
.is_some_and(|captures| captures.is_track_path(path));
|
||||||
}
|
}
|
||||||
|
if youtube_owns(path) {
|
||||||
|
return self
|
||||||
|
.youtube_client
|
||||||
|
.as_ref()
|
||||||
|
.is_some_and(|youtube| youtube.is_track_path(path));
|
||||||
|
}
|
||||||
false
|
false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -391,6 +432,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.get_urls_for_track(track_path)
|
.get_urls_for_track(track_path)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
if youtube_owns(track_path) {
|
||||||
|
return self
|
||||||
|
.youtube_provider()?
|
||||||
|
.get_urls_for_track(track_path)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
warn!(path = track_path, "no provider owns this track path");
|
warn!(path = track_path, "no provider owns this track path");
|
||||||
Err(ProviderError::MalformedPath)
|
Err(ProviderError::MalformedPath)
|
||||||
}
|
}
|
||||||
|
|
@ -421,6 +468,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.get_metadata_for_track(track_path)
|
.get_metadata_for_track(track_path)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
if youtube_owns(track_path) {
|
||||||
|
return self
|
||||||
|
.youtube_provider()?
|
||||||
|
.get_metadata_for_track(track_path)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
warn!(path = track_path, "no provider owns this track path");
|
warn!(path = track_path, "no provider owns this track path");
|
||||||
Err(ProviderError::MalformedPath)
|
Err(ProviderError::MalformedPath)
|
||||||
}
|
}
|
||||||
|
|
@ -456,6 +509,11 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
);
|
);
|
||||||
root_node.children.push(child);
|
root_node.children.push(child);
|
||||||
}
|
}
|
||||||
|
if self.youtube_client.is_some() {
|
||||||
|
let child =
|
||||||
|
LibraryNodeChild::new(ytdy::PROVIDER_ROOT.to_owned(), "youtube".to_owned(), false);
|
||||||
|
root_node.children.push(child);
|
||||||
|
}
|
||||||
root_node
|
root_node
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -480,6 +538,9 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
if captures_owns(path) {
|
if captures_owns(path) {
|
||||||
return self.captures_provider()?.get_lib_node(path).await;
|
return self.captures_provider()?.get_lib_node(path).await;
|
||||||
}
|
}
|
||||||
|
if youtube_owns(path) {
|
||||||
|
return self.youtube_provider()?.get_lib_node(path).await;
|
||||||
|
}
|
||||||
warn!(path, "no provider owns this path");
|
warn!(path, "no provider owns this path");
|
||||||
Err(ProviderError::MalformedPath)
|
Err(ProviderError::MalformedPath)
|
||||||
}
|
}
|
||||||
|
|
@ -519,6 +580,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.create_lib_node(parent_path, title)
|
.create_lib_node(parent_path, title)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
if youtube_owns(parent_path) {
|
||||||
|
return self
|
||||||
|
.youtube_provider()?
|
||||||
|
.create_lib_node(parent_path, title)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
warn!(parent_path, "no provider supports creating nodes here");
|
warn!(parent_path, "no provider supports creating nodes here");
|
||||||
Err(ProviderError::NotSupported)
|
Err(ProviderError::NotSupported)
|
||||||
}
|
}
|
||||||
|
|
@ -555,6 +622,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.rename_lib_node(path, new_title)
|
.rename_lib_node(path, new_title)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
if youtube_owns(path) {
|
||||||
|
return self
|
||||||
|
.youtube_provider()?
|
||||||
|
.rename_lib_node(path, new_title)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
warn!(path, "no provider supports renaming this node");
|
warn!(path, "no provider supports renaming this node");
|
||||||
Err(ProviderError::NotSupported)
|
Err(ProviderError::NotSupported)
|
||||||
}
|
}
|
||||||
|
|
@ -594,6 +667,12 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
.resolve_tracks_into(path, chunk_tx)
|
.resolve_tracks_into(path, chunk_tx)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
|
if youtube_owns(path) {
|
||||||
|
return self
|
||||||
|
.youtube_provider()?
|
||||||
|
.resolve_tracks_into(path, chunk_tx)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
warn!(path, "no provider owns this path");
|
warn!(path, "no provider owns this path");
|
||||||
Err(ProviderError::MalformedPath)
|
Err(ProviderError::MalformedPath)
|
||||||
}
|
}
|
||||||
|
|
@ -617,6 +696,9 @@ impl ProviderClient for ProviderOrchestrator {
|
||||||
if captures_owns(path) {
|
if captures_owns(path) {
|
||||||
return self.captures_provider()?.delete_lib_node(path).await;
|
return self.captures_provider()?.delete_lib_node(path).await;
|
||||||
}
|
}
|
||||||
|
if youtube_owns(path) {
|
||||||
|
return self.youtube_provider()?.delete_lib_node(path).await;
|
||||||
|
}
|
||||||
warn!(path, "no provider supports deleting this node");
|
warn!(path, "no provider supports deleting this node");
|
||||||
Err(ProviderError::NotSupported)
|
Err(ProviderError::NotSupported)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -14,6 +14,8 @@ let
|
||||||
d2
|
d2
|
||||||
pkg-config
|
pkg-config
|
||||||
protobuf
|
protobuf
|
||||||
|
# Extraction engine for the ytdy provider (architecture/youtube-provider.md D1).
|
||||||
|
yt-dlp
|
||||||
];
|
];
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,38 @@
|
||||||
# Implementation summaries
|
# Implementation summaries
|
||||||
|
|
||||||
|
## youtube-provider (2026-07-21)
|
||||||
|
|
||||||
|
Built per `plan/youtube-provider.md`: a new workspace crate **`ytdy`**
|
||||||
|
mounts YouTube at `/youtube`, backed by a `yt-dlp` subprocess (declared
|
||||||
|
in `devenv.nix`). All extraction goes through one `Engine` seam:
|
||||||
|
argv-only invocations with `--no-warnings`, an optional `--cookies`
|
||||||
|
flag, a per-call timeout (`kill_on_drop`), a 32 MiB stdout cap, and
|
||||||
|
typed `EngineError`s — tests drive the whole provider through a fake
|
||||||
|
shell-script binary, no network.
|
||||||
|
|
||||||
|
Search needs no login and mirrors tidal's search exactly: `%` on
|
||||||
|
`/youtube/search` creates an in-memory term (deduplicated, implicitly
|
||||||
|
recreated on stale paths, rename re-searches, delete idempotent), whose
|
||||||
|
node lists the top N (`ytsearchN:`, default 20) results as queueable,
|
||||||
|
downloadable tracks. With a readable cookies file configured in
|
||||||
|
`ytdy.toml` ("logged in"), a `playlists` subtree appears
|
||||||
|
(`feed/playlists` flat listing → playlist nodes with tracks); an
|
||||||
|
unreadable cookies file degrades to logged-out with a warning, never a
|
||||||
|
failed init. Streams resolve via `-f bestaudio/best -g`; captures work
|
||||||
|
end to end (`extension_for` gained `audio/webm → webm`). The
|
||||||
|
orchestrator wires `/youtube` non-fatally: a failed `--version` probe
|
||||||
|
disables the provider, nothing else.
|
||||||
|
|
||||||
|
Deviations: `Entry` keeps separate `uploader`/`channel` fields with an
|
||||||
|
`artist()` preference — the planned serde alias rejects real yt-dlp
|
||||||
|
output as a duplicate field (found by the live probe). The live probe
|
||||||
|
validated search, stream resolution, and a real download capture
|
||||||
|
(252 KB webm) through the capture store; the **playlists feed
|
||||||
|
invocation is live-unvalidated** (no cookies on this machine) — flagged
|
||||||
|
in `architecture/youtube-provider.md` as the standing risk. 150
|
||||||
|
workspace tests green (8 new in `ytdy`); every gate in
|
||||||
|
`quality/youtube-provider.md` checked.
|
||||||
|
|
||||||
## captures (2026-07-21)
|
## captures (2026-07-21)
|
||||||
|
|
||||||
Built per `plan/captures.md`: `W` (shift) on a downloadable library
|
Built per `plan/captures.md`: `W` (shift) on a downloadable library
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,45 @@
|
||||||
|
# Plan: youtube-provider
|
||||||
|
|
||||||
|
Ordered tasks; each names its verification (tests in `ytdy/src/lib.rs`
|
||||||
|
and/or gates in `quality/youtube-provider.md`). The crate skeleton,
|
||||||
|
stubs, workspace/devenv wiring, and all tests exist; the tests fail on
|
||||||
|
`todo!()` at plan time.
|
||||||
|
|
||||||
|
- [x] **T1 — Engine.** `run` (argv-only, `--no-warnings`, cookie flag,
|
||||||
|
timeout with `kill_on_drop`, stdout cap, typed errors), `probe`,
|
||||||
|
`flat_listing`, `video_entry`, `stream_urls`. Verifies:
|
||||||
|
`engine_failures_are_typed_never_panics`, parts of every other test;
|
||||||
|
gates "Engine".
|
||||||
|
- [x] **T2 — Path parsing.** `parse_path` for all `YtPath` shapes;
|
||||||
|
reject everything else. Verifies:
|
||||||
|
`foreign_and_malformed_paths_are_rejected`,
|
||||||
|
`tracks_resolve_streams_and_metadata` (is_track_path).
|
||||||
|
- [x] **T3 — Init.** Settings parse (defaults on broken TOML like the
|
||||||
|
other providers), engine construction, `--version` probe, cookies
|
||||||
|
readability check (degrade to logged out with a warning). Verifies:
|
||||||
|
`init_probes_the_binary`, `root_lists_playlists_only_when_logged_in`.
|
||||||
|
- [x] **T4 — Search.** Term registry (snapshot/register),
|
||||||
|
`search_term_node` (`ytsearchN:`), `create/rename/delete_lib_node`,
|
||||||
|
`get_lib_node` arms for Root/Search/SearchTerm, central
|
||||||
|
downloadable rule. Verifies:
|
||||||
|
`search_terms_are_created_listed_and_searched`,
|
||||||
|
`search_terms_rename_and_delete`.
|
||||||
|
- [x] **T5 — Playlists + tracks.** `playlists_node` (feed URL, login
|
||||||
|
gate), `playlist_node`, `get_urls_for_track` (`-g`),
|
||||||
|
`get_metadata_for_track` (single-video `-J`). Verifies:
|
||||||
|
`playlists_list_and_resolve_when_logged_in`,
|
||||||
|
`tracks_resolve_streams_and_metadata`.
|
||||||
|
- [x] **T6 — Orchestrator wiring.** `youtube_client: Option<Arc<_>>`,
|
||||||
|
non-fatal init from `ytdy.toml` (written back), routing arms in every
|
||||||
|
method, root child; `extension_for` gains `audio/webm`. Verifies:
|
||||||
|
existing orchestrator patterns by inspection; gates "Orchestrator
|
||||||
|
wiring"; `extension_prefers_content_type_then_url_then_bin` extended.
|
||||||
|
- [x] **T7 — Full verification.** Workspace suite green; clippy/fmt/
|
||||||
|
taplo/markdownlint clean; every gate ticked; no `todo!()`.
|
||||||
|
- [x] **T8 — Live smoke test.** With the devenv `yt-dlp`: search a term
|
||||||
|
through the provider, fetch a stream URL for one result, and download
|
||||||
|
a small capture through the capture store — remove the probe
|
||||||
|
afterwards. Validate the playlists feed invocation if cookies are
|
||||||
|
available; otherwise record it as unvalidated in the summary.
|
||||||
|
- [x] **T9 — Docs.** `plan/summary.md` section incl. deviations;
|
||||||
|
reconcile `architecture/youtube-provider.md`.
|
||||||
|
|
@ -0,0 +1,57 @@
|
||||||
|
# Quality gates: youtube-provider
|
||||||
|
|
||||||
|
Criteria beyond the automatic tests (`ytdy/src/lib.rs`,
|
||||||
|
`ytdy/src/engine.rs`, plus orchestrator wiring). Each gate is pass/fail
|
||||||
|
by reading the code.
|
||||||
|
|
||||||
|
## Engine (subprocess discipline)
|
||||||
|
|
||||||
|
- [x] Every invocation is an argument list (`tokio::process::Command`)
|
||||||
|
— never a shell string; user input (terms, ids) can contain anything.
|
||||||
|
- [x] Every call is bounded: per-call timeout (`kill_on_drop` so a timed
|
||||||
|
out process dies), and captured stdout capped at `MAX_STDOUT_BYTES`.
|
||||||
|
- [x] All failures are typed `EngineError`s mapped to `ProviderError`
|
||||||
|
at the trait boundary; no subprocess condition panics.
|
||||||
|
- [x] Error/log output carries the binary path, exit status, and a
|
||||||
|
*bounded* stderr summary — never full stderr, never stream URLs,
|
||||||
|
never cookie file contents (the path alone is loggable).
|
||||||
|
- [x] The cookies flag is appended to every call when configured;
|
||||||
|
nothing else about login is stored or invented.
|
||||||
|
|
||||||
|
## Provider semantics
|
||||||
|
|
||||||
|
- [x] Search terms mirror tidal's: in-memory, deduplicated, implicit
|
||||||
|
recreation on stale paths, rename re-searches, delete idempotent —
|
||||||
|
and only `/youtube/search` children are creatable/editable/deletable.
|
||||||
|
- [x] Path parsing rejects foreign roots and malformed shapes with
|
||||||
|
`MalformedPath`; track paths are exactly `<node>/<videoid>`.
|
||||||
|
- [x] Downloadability follows the central rule (queueable or lists
|
||||||
|
tracks; children mirror `is_queable`) applied in one place, not
|
||||||
|
per-arm.
|
||||||
|
- [x] `duration` floats are truncated to whole seconds; missing
|
||||||
|
title/uploader degrade to empty strings, never errors.
|
||||||
|
- [x] The playlists subtree exists only when logged in: absent from the
|
||||||
|
root listing and `MalformedPath` when addressed directly.
|
||||||
|
|
||||||
|
## Orchestrator wiring
|
||||||
|
|
||||||
|
- [x] `ytdy::Client::init` failure (missing/broken binary) disables the
|
||||||
|
provider with a warning — the server and every other provider keep
|
||||||
|
running; `get_lib_root` lists `youtube` only when enabled.
|
||||||
|
- [x] `/youtube` is routed in every `ProviderClient` method (same
|
||||||
|
completeness as `/tidal`), including `resolve_tracks_into`,
|
||||||
|
create/rename/delete.
|
||||||
|
- [x] `ytdy.toml` is written back after init like the other providers'
|
||||||
|
configs.
|
||||||
|
- [x] `extension_for` maps `audio/webm → webm` so captures of YouTube
|
||||||
|
audio get a sensible extension.
|
||||||
|
|
||||||
|
## Hygiene
|
||||||
|
|
||||||
|
- [x] `yt-dlp` is declared in `devenv.nix`, never assumed installed.
|
||||||
|
- [x] New public items are documented; docs state error/edge behavior.
|
||||||
|
- [x] `clippy -D warnings`, `fmt`, `taplo`, `markdownlint` clean; all
|
||||||
|
tests green; no `todo!()` left.
|
||||||
|
- [x] `architecture/youtube-provider.md` reconciled where the
|
||||||
|
implementation diverged (esp. the playlists-feed invocation after the
|
||||||
|
live probe).
|
||||||
|
|
@ -0,0 +1,24 @@
|
||||||
|
[package]
|
||||||
|
name = "ytdy"
|
||||||
|
version.workspace = true
|
||||||
|
edition.workspace = true
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
async-trait.workspace = true
|
||||||
|
crabidy-core.workspace = true
|
||||||
|
serde.workspace = true
|
||||||
|
serde_json.workspace = true
|
||||||
|
thiserror.workspace = true
|
||||||
|
tokio = { workspace = true, features = [
|
||||||
|
"time",
|
||||||
|
"sync",
|
||||||
|
"macros",
|
||||||
|
"process",
|
||||||
|
"io-util",
|
||||||
|
] }
|
||||||
|
toml.workspace = true
|
||||||
|
tracing.workspace = true
|
||||||
|
|
||||||
|
[dev-dependencies]
|
||||||
|
tempfile.workspace = true
|
||||||
|
tokio = { workspace = true, features = ["full"] }
|
||||||
|
|
@ -0,0 +1,178 @@
|
||||||
|
//! The yt-dlp subprocess seam (see `architecture/youtube-provider.md` D1/D5).
|
||||||
|
//!
|
||||||
|
//! Every YouTube interaction is one bounded `yt-dlp` invocation: argument
|
||||||
|
//! list only (never a shell), `--no-warnings`, an optional `--cookies`
|
||||||
|
//! file, a per-call timeout, and a cap on captured stdout. JSON output is
|
||||||
|
//! parsed with `serde_json`; failures are typed [`EngineError`]s — no
|
||||||
|
//! subprocess condition may panic. Tests point `binary` at a fake script.
|
||||||
|
|
||||||
|
use std::path::PathBuf;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use serde::Deserialize;
|
||||||
|
|
||||||
|
/// Captured stdout beyond this many bytes aborts the call — a runaway
|
||||||
|
/// extractor must not balloon memory.
|
||||||
|
pub const MAX_STDOUT_BYTES: usize = 32 * 1024 * 1024;
|
||||||
|
|
||||||
|
/// Errors from one engine invocation.
|
||||||
|
///
|
||||||
|
/// Messages carry the binary name, exit status, and a short stderr
|
||||||
|
/// summary — never full stderr (it can embed URLs) and never cookie
|
||||||
|
/// contents.
|
||||||
|
#[derive(Debug, thiserror::Error)]
|
||||||
|
pub enum EngineError {
|
||||||
|
#[error("cannot run yt-dlp: {0}")]
|
||||||
|
Spawn(#[from] std::io::Error),
|
||||||
|
#[error("yt-dlp timed out after {0:?}")]
|
||||||
|
Timeout(Duration),
|
||||||
|
#[error("yt-dlp failed ({status}): {summary}")]
|
||||||
|
Failed { status: String, summary: String },
|
||||||
|
#[error("cannot parse yt-dlp output: {0}")]
|
||||||
|
Parse(#[from] serde_json::Error),
|
||||||
|
#[error("yt-dlp output exceeded {MAX_STDOUT_BYTES} bytes")]
|
||||||
|
Oversized,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One entry of a `--flat-playlist -J` listing (or a single-video `-J`).
|
||||||
|
/// Fields we don't use are ignored; absent fields stay `None` — flat
|
||||||
|
/// extraction fills what it cheaply can.
|
||||||
|
#[derive(Debug, Clone, Deserialize)]
|
||||||
|
pub struct Entry {
|
||||||
|
pub id: String,
|
||||||
|
#[serde(default)]
|
||||||
|
pub title: Option<String>,
|
||||||
|
/// Real yt-dlp output carries `uploader` *and* `channel` (so a serde
|
||||||
|
/// alias would reject it as a duplicate field); both are kept and
|
||||||
|
/// [`Self::artist`] picks.
|
||||||
|
#[serde(default)]
|
||||||
|
pub uploader: Option<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub channel: Option<String>,
|
||||||
|
/// Seconds; yt-dlp emits floats.
|
||||||
|
#[serde(default)]
|
||||||
|
pub duration: Option<f64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Entry {
|
||||||
|
/// The track artist: the uploader, falling back to the channel.
|
||||||
|
pub fn artist(&self) -> Option<&str> {
|
||||||
|
self.uploader.as_deref().or(self.channel.as_deref())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A `-J` playlist-shaped result: its `entries`, plus the playlist title.
|
||||||
|
#[derive(Debug, Deserialize)]
|
||||||
|
pub struct Listing {
|
||||||
|
#[serde(default)]
|
||||||
|
pub title: Option<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub entries: Vec<Entry>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The shared subprocess runner. Cheap to clone-by-reference (the client
|
||||||
|
/// holds one); all state is configuration.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub struct Engine {
|
||||||
|
binary: PathBuf,
|
||||||
|
cookies: Option<PathBuf>,
|
||||||
|
timeout: Duration,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Engine {
|
||||||
|
/// An engine invoking `binary` with `timeout` per call, passing
|
||||||
|
/// `--cookies <cookies>` on every call when set.
|
||||||
|
pub fn new(binary: PathBuf, cookies: Option<PathBuf>, timeout: Duration) -> Self {
|
||||||
|
Self {
|
||||||
|
binary,
|
||||||
|
cookies,
|
||||||
|
timeout,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `yt-dlp --version` — the init probe. The trimmed version string on
|
||||||
|
/// success.
|
||||||
|
pub async fn probe(&self) -> Result<String, EngineError> {
|
||||||
|
let stdout = self.run(&["--version"]).await?;
|
||||||
|
Ok(String::from_utf8_lossy(&stdout).trim().to_string())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A flat listing (`-J --flat-playlist`) of `target` — a
|
||||||
|
/// `ytsearchN:` term, a playlist URL, or the playlists feed.
|
||||||
|
pub async fn flat_listing(&self, target: &str) -> Result<Listing, EngineError> {
|
||||||
|
let stdout = self.run(&["-J", "--flat-playlist", target]).await?;
|
||||||
|
Ok(serde_json::from_slice(&stdout)?)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Metadata of one video (`-J --no-playlist`).
|
||||||
|
pub async fn video_entry(&self, video_url: &str) -> Result<Entry, EngineError> {
|
||||||
|
let stdout = self.run(&["-J", "--no-playlist", video_url]).await?;
|
||||||
|
Ok(serde_json::from_slice(&stdout)?)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stream URL(s) of one video (`-f bestaudio/best -g --no-playlist`),
|
||||||
|
/// one per line.
|
||||||
|
pub async fn stream_urls(&self, video_url: &str) -> Result<Vec<String>, EngineError> {
|
||||||
|
let stdout = self
|
||||||
|
.run(&["-f", "bestaudio/best", "-g", "--no-playlist", video_url])
|
||||||
|
.await?;
|
||||||
|
Ok(String::from_utf8_lossy(&stdout)
|
||||||
|
.lines()
|
||||||
|
.map(str::trim)
|
||||||
|
.filter(|line| !line.is_empty())
|
||||||
|
.map(str::to_string)
|
||||||
|
.collect())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Runs the binary with `args` (plus `--no-warnings` and the cookie
|
||||||
|
/// flag), enforcing the timeout and stdout cap; returns raw stdout.
|
||||||
|
///
|
||||||
|
/// The child is `kill_on_drop`: hitting the timeout kills it instead
|
||||||
|
/// of leaking a hung extractor.
|
||||||
|
async fn run(&self, args: &[&str]) -> Result<Vec<u8>, EngineError> {
|
||||||
|
use std::process::Stdio;
|
||||||
|
use tokio::io::AsyncReadExt;
|
||||||
|
|
||||||
|
let mut cmd = tokio::process::Command::new(&self.binary);
|
||||||
|
cmd.arg("--no-warnings");
|
||||||
|
if let Some(cookies) = &self.cookies {
|
||||||
|
cmd.arg("--cookies").arg(cookies);
|
||||||
|
}
|
||||||
|
cmd.args(args)
|
||||||
|
.stdin(Stdio::null())
|
||||||
|
.stdout(Stdio::piped())
|
||||||
|
.stderr(Stdio::piped())
|
||||||
|
.kill_on_drop(true);
|
||||||
|
let mut child = cmd.spawn()?;
|
||||||
|
// Read stdout incrementally with a hard cap — `wait_with_output`
|
||||||
|
// alone would buffer an unbounded stream first.
|
||||||
|
let mut stdout_pipe = child.stdout.take().ok_or_else(|| {
|
||||||
|
std::io::Error::other("child stdout pipe missing despite Stdio::piped")
|
||||||
|
})?;
|
||||||
|
let bounded = async {
|
||||||
|
let mut stdout = Vec::new();
|
||||||
|
let mut limited = (&mut stdout_pipe).take(MAX_STDOUT_BYTES as u64 + 1);
|
||||||
|
limited.read_to_end(&mut stdout).await?;
|
||||||
|
let rest = child.wait_with_output().await?;
|
||||||
|
Ok::<_, std::io::Error>((stdout, rest))
|
||||||
|
};
|
||||||
|
let (stdout, rest) = tokio::time::timeout(self.timeout, bounded)
|
||||||
|
.await
|
||||||
|
.map_err(|_| EngineError::Timeout(self.timeout))??;
|
||||||
|
if stdout.len() > MAX_STDOUT_BYTES {
|
||||||
|
return Err(EngineError::Oversized);
|
||||||
|
}
|
||||||
|
if !rest.status.success() {
|
||||||
|
// A bounded summary only: full stderr can embed URLs.
|
||||||
|
let summary: String = String::from_utf8_lossy(&rest.stderr)
|
||||||
|
.chars()
|
||||||
|
.take(200)
|
||||||
|
.collect();
|
||||||
|
return Err(EngineError::Failed {
|
||||||
|
status: rest.status.to_string(),
|
||||||
|
summary: summary.trim().to_string(),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
Ok(stdout)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,806 @@
|
||||||
|
//! YouTube media provider, backed by a `yt-dlp` subprocess
|
||||||
|
//! (see `architecture/youtube-provider.md`).
|
||||||
|
//!
|
||||||
|
//! Mounted at [`PROVIDER_ROOT`]. Search works without login: creatable
|
||||||
|
//! search-term nodes exactly like `/tidal/search` (in-memory terms,
|
||||||
|
//! renamable/deletable, results listed as tracks). With a cookies file
|
||||||
|
//! configured ("logged in"), the user's playlists appear under
|
||||||
|
//! `/youtube/playlists`. Every node that serves tracks is downloadable —
|
||||||
|
//! `W` captures work out of the box.
|
||||||
|
|
||||||
|
use std::path::PathBuf;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use async_trait::async_trait;
|
||||||
|
use crabidy_core::proto::crabidy::{LibraryNode, LibraryNodeChild, Track};
|
||||||
|
use crabidy_core::{ProviderClient, ProviderError};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use tracing::{debug, warn};
|
||||||
|
|
||||||
|
pub mod engine;
|
||||||
|
use engine::Engine;
|
||||||
|
|
||||||
|
/// First path segment owned by this provider.
|
||||||
|
pub const PROVIDER_ROOT: &str = "/youtube";
|
||||||
|
|
||||||
|
/// Default number of search results per term (`ytsearchN:`).
|
||||||
|
pub const DEFAULT_SEARCH_RESULTS: usize = 20;
|
||||||
|
|
||||||
|
/// Default per-subprocess-call timeout in seconds.
|
||||||
|
pub const DEFAULT_CALL_TIMEOUT_SECS: u64 = 60;
|
||||||
|
|
||||||
|
/// Provider settings, persisted as `ytdy.toml` next to the other crabidy
|
||||||
|
/// config files. The cookies file is user-provided and its **contents are
|
||||||
|
/// a secret**: only the path may ever be logged.
|
||||||
|
#[derive(Clone, Debug, Default, Deserialize, Serialize)]
|
||||||
|
pub struct Settings {
|
||||||
|
/// The yt-dlp binary; a bare name resolves via PATH. Default
|
||||||
|
/// `yt-dlp`.
|
||||||
|
pub binary: Option<PathBuf>,
|
||||||
|
/// Netscape cookies file for logged-in features (playlists,
|
||||||
|
/// age-gated streams). Absent = logged out; search still works.
|
||||||
|
pub cookies: Option<PathBuf>,
|
||||||
|
/// Results per search term. Default [`DEFAULT_SEARCH_RESULTS`].
|
||||||
|
pub search_results: Option<usize>,
|
||||||
|
/// Per-call timeout in seconds. Default
|
||||||
|
/// [`DEFAULT_CALL_TIMEOUT_SECS`]; tests shrink it.
|
||||||
|
pub call_timeout_secs: Option<u64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A parsed `/youtube/...` path.
|
||||||
|
#[derive(Debug, PartialEq, Eq)]
|
||||||
|
enum YtPath<'a> {
|
||||||
|
Root,
|
||||||
|
Search,
|
||||||
|
/// Percent-encoded search term segment.
|
||||||
|
SearchTerm(&'a str),
|
||||||
|
SearchTrack {
|
||||||
|
term: &'a str,
|
||||||
|
video: &'a str,
|
||||||
|
},
|
||||||
|
Playlists,
|
||||||
|
Playlist(&'a str),
|
||||||
|
PlaylistTrack {
|
||||||
|
playlist: &'a str,
|
||||||
|
video: &'a str,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Splits a `/youtube/...` path into its recognized shape.
|
||||||
|
/// Unknown shapes are [`ProviderError::MalformedPath`].
|
||||||
|
fn parse_path(path: &str) -> Result<YtPath<'_>, ProviderError> {
|
||||||
|
if path == PROVIDER_ROOT {
|
||||||
|
return Ok(YtPath::Root);
|
||||||
|
}
|
||||||
|
let rest = path
|
||||||
|
.strip_prefix("/youtube/")
|
||||||
|
.ok_or(ProviderError::MalformedPath)?;
|
||||||
|
let segments: Vec<&str> = rest.split('/').collect();
|
||||||
|
if segments.iter().any(|segment| segment.is_empty()) {
|
||||||
|
return Err(ProviderError::MalformedPath);
|
||||||
|
}
|
||||||
|
match segments.as_slice() {
|
||||||
|
["search"] => Ok(YtPath::Search),
|
||||||
|
["search", term] => Ok(YtPath::SearchTerm(term)),
|
||||||
|
["search", term, video] => Ok(YtPath::SearchTrack { term, video }),
|
||||||
|
["playlists"] => Ok(YtPath::Playlists),
|
||||||
|
["playlists", playlist] => Ok(YtPath::Playlist(playlist)),
|
||||||
|
["playlists", playlist, video] => Ok(YtPath::PlaylistTrack { playlist, video }),
|
||||||
|
_ => Err(ProviderError::MalformedPath),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Maps an engine failure to the trait-level error, logging the typed
|
||||||
|
/// cause (paths and statuses only — never URLs or cookie contents).
|
||||||
|
fn engine_err(context: &str, err: engine::EngineError) -> ProviderError {
|
||||||
|
warn!(context, "yt-dlp call failed: {err}");
|
||||||
|
ProviderError::FetchError
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The canonical watch URL for a video id.
|
||||||
|
fn video_url(video_id: &str) -> String {
|
||||||
|
format!("https://www.youtube.com/watch?v={video_id}")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Builds the wire track for one listing entry under `node_path`.
|
||||||
|
/// Missing metadata degrades to empty fields, never an error; float
|
||||||
|
/// durations truncate to whole seconds.
|
||||||
|
fn entry_to_track(entry: &engine::Entry, node_path: &str) -> Track {
|
||||||
|
Track {
|
||||||
|
path: crabidy_core::join_path(node_path, &entry.id),
|
||||||
|
artist: entry.artist().unwrap_or_default().to_string(),
|
||||||
|
title: entry.title.clone().unwrap_or_default(),
|
||||||
|
duration: entry.duration.map(|secs| secs.max(0.0) as u32),
|
||||||
|
album: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The YouTube provider client.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub struct Client {
|
||||||
|
engine: Engine,
|
||||||
|
settings: Settings,
|
||||||
|
/// Cookies were configured and readable at init — gates the
|
||||||
|
/// playlists subtree.
|
||||||
|
logged_in: bool,
|
||||||
|
/// Search terms created under `/youtube/search`, in creation order,
|
||||||
|
/// deduplicated. In-memory only, like tidal's. Never held across
|
||||||
|
/// awaits.
|
||||||
|
search_terms: std::sync::RwLock<Vec<String>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Client {
|
||||||
|
/// The search-term node: top `search_results` results as tracks
|
||||||
|
/// (`ytsearchN:<term>` flat listing). Queueable and downloadable —
|
||||||
|
/// results are homogeneous tracks.
|
||||||
|
async fn search_term_node(
|
||||||
|
&self,
|
||||||
|
path: &str,
|
||||||
|
term: &str,
|
||||||
|
parent: String,
|
||||||
|
) -> Result<LibraryNode, ProviderError> {
|
||||||
|
let count = self
|
||||||
|
.settings
|
||||||
|
.search_results
|
||||||
|
.unwrap_or(DEFAULT_SEARCH_RESULTS);
|
||||||
|
let listing = self
|
||||||
|
.engine
|
||||||
|
.flat_listing(&format!("ytsearch{count}:{term}"))
|
||||||
|
.await
|
||||||
|
.map_err(|err| engine_err("search", err))?;
|
||||||
|
Ok(LibraryNode {
|
||||||
|
path: path.to_string(),
|
||||||
|
title: term.to_string(),
|
||||||
|
parent: Some(parent),
|
||||||
|
tracks: listing
|
||||||
|
.entries
|
||||||
|
.iter()
|
||||||
|
.map(|entry| entry_to_track(entry, path))
|
||||||
|
.collect(),
|
||||||
|
children: Vec::new(),
|
||||||
|
is_queable: true,
|
||||||
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The playlists listing (`feed/playlists`, cookies required):
|
||||||
|
/// one queueable, downloadable child per playlist.
|
||||||
|
async fn playlists_node(
|
||||||
|
&self,
|
||||||
|
path: &str,
|
||||||
|
parent: String,
|
||||||
|
) -> Result<LibraryNode, ProviderError> {
|
||||||
|
let listing = self
|
||||||
|
.engine
|
||||||
|
.flat_listing("https://www.youtube.com/feed/playlists")
|
||||||
|
.await
|
||||||
|
.map_err(|err| engine_err("playlists feed", err))?;
|
||||||
|
Ok(LibraryNode {
|
||||||
|
path: path.to_string(),
|
||||||
|
title: "playlists".to_string(),
|
||||||
|
parent: Some(parent),
|
||||||
|
tracks: Vec::new(),
|
||||||
|
children: listing
|
||||||
|
.entries
|
||||||
|
.iter()
|
||||||
|
.map(|entry| {
|
||||||
|
LibraryNodeChild::new(
|
||||||
|
crabidy_core::join_path(path, &entry.id),
|
||||||
|
entry.title.clone().unwrap_or_else(|| entry.id.clone()),
|
||||||
|
true,
|
||||||
|
)
|
||||||
|
})
|
||||||
|
.collect(),
|
||||||
|
is_queable: false,
|
||||||
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One playlist's entries as tracks.
|
||||||
|
async fn playlist_node(
|
||||||
|
&self,
|
||||||
|
path: &str,
|
||||||
|
playlist_id: &str,
|
||||||
|
parent: String,
|
||||||
|
) -> Result<LibraryNode, ProviderError> {
|
||||||
|
let listing = self
|
||||||
|
.engine
|
||||||
|
.flat_listing(&format!(
|
||||||
|
"https://www.youtube.com/playlist?list={playlist_id}"
|
||||||
|
))
|
||||||
|
.await
|
||||||
|
.map_err(|err| engine_err("playlist", err))?;
|
||||||
|
Ok(LibraryNode {
|
||||||
|
path: path.to_string(),
|
||||||
|
title: listing
|
||||||
|
.title
|
||||||
|
.clone()
|
||||||
|
.unwrap_or_else(|| playlist_id.to_string()),
|
||||||
|
parent: Some(parent),
|
||||||
|
tracks: listing
|
||||||
|
.entries
|
||||||
|
.iter()
|
||||||
|
.map(|entry| entry_to_track(entry, path))
|
||||||
|
.collect(),
|
||||||
|
children: Vec::new(),
|
||||||
|
is_queable: true,
|
||||||
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn search_terms_snapshot(&self) -> Vec<String> {
|
||||||
|
self.search_terms
|
||||||
|
.read()
|
||||||
|
.map(|terms| terms.clone())
|
||||||
|
.unwrap_or_default()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn register_search_term(&self, term: &str) {
|
||||||
|
if let Ok(mut terms) = self.search_terms.write() {
|
||||||
|
if !terms.iter().any(|existing| existing == term) {
|
||||||
|
terms.push(term.to_string());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Removes a term; `true` when it existed.
|
||||||
|
fn remove_search_term(&self, term: &str) -> bool {
|
||||||
|
match self.search_terms.write() {
|
||||||
|
Ok(mut terms) => {
|
||||||
|
let before = terms.len();
|
||||||
|
terms.retain(|existing| existing != term);
|
||||||
|
terms.len() != before
|
||||||
|
}
|
||||||
|
Err(_) => false,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl ProviderClient for Client {
|
||||||
|
/// Builds the engine from settings and probes `--version`; a missing
|
||||||
|
/// or broken binary fails init (the orchestrator disables the
|
||||||
|
/// provider non-fatally). A configured but unreadable cookies file
|
||||||
|
/// degrades to logged-out with a warning, never an error.
|
||||||
|
async fn init(raw_toml_settings: &str) -> Result<Self, ProviderError> {
|
||||||
|
let settings: Settings = toml::from_str(raw_toml_settings).unwrap_or_else(|_| {
|
||||||
|
warn!("could not parse toml settings, using defaults");
|
||||||
|
Settings::default()
|
||||||
|
});
|
||||||
|
let binary = settings
|
||||||
|
.binary
|
||||||
|
.clone()
|
||||||
|
.unwrap_or_else(|| PathBuf::from("yt-dlp"));
|
||||||
|
let timeout = Duration::from_secs(
|
||||||
|
settings
|
||||||
|
.call_timeout_secs
|
||||||
|
.unwrap_or(DEFAULT_CALL_TIMEOUT_SECS),
|
||||||
|
);
|
||||||
|
// "Logged in" is exactly "the configured cookies file is
|
||||||
|
// readable"; anything less degrades to logged out, never to a
|
||||||
|
// failed init. Only the *path* is ever logged.
|
||||||
|
let logged_in = match &settings.cookies {
|
||||||
|
Some(path) => match std::fs::metadata(path) {
|
||||||
|
Ok(meta) if meta.is_file() => true,
|
||||||
|
_ => {
|
||||||
|
warn!(
|
||||||
|
cookies = %path.display(),
|
||||||
|
"cookies file not readable; running logged out"
|
||||||
|
);
|
||||||
|
false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
None => false,
|
||||||
|
};
|
||||||
|
let cookies = logged_in.then(|| settings.cookies.clone()).flatten();
|
||||||
|
let engine = Engine::new(binary, cookies, timeout);
|
||||||
|
let version = engine.probe().await.map_err(|err| {
|
||||||
|
warn!("yt-dlp probe failed: {err}");
|
||||||
|
ProviderError::Config(err.to_string())
|
||||||
|
})?;
|
||||||
|
debug!(version, logged_in, "yt-dlp ready");
|
||||||
|
Ok(Self {
|
||||||
|
engine,
|
||||||
|
settings,
|
||||||
|
logged_in,
|
||||||
|
search_terms: std::sync::RwLock::new(Vec::new()),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn settings(&self) -> String {
|
||||||
|
toml::to_string_pretty(&self.settings).unwrap_or_default()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_track_path(&self, path: &str) -> bool {
|
||||||
|
matches!(
|
||||||
|
parse_path(path),
|
||||||
|
Ok(YtPath::SearchTrack { .. } | YtPath::PlaylistTrack { .. })
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `-f bestaudio/best -g` on the track's video id.
|
||||||
|
async fn get_urls_for_track(&self, track_path: &str) -> Result<Vec<String>, ProviderError> {
|
||||||
|
let video = match parse_path(track_path)? {
|
||||||
|
YtPath::SearchTrack { video, .. } | YtPath::PlaylistTrack { video, .. } => video,
|
||||||
|
_ => return Err(ProviderError::MalformedPath),
|
||||||
|
};
|
||||||
|
let urls = self
|
||||||
|
.engine
|
||||||
|
.stream_urls(&video_url(video))
|
||||||
|
.await
|
||||||
|
.map_err(|err| engine_err("stream urls", err))?;
|
||||||
|
if urls.is_empty() {
|
||||||
|
warn!(path = track_path, "yt-dlp returned no stream url");
|
||||||
|
return Err(ProviderError::FetchError);
|
||||||
|
}
|
||||||
|
Ok(urls)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Single-video `-J` metadata.
|
||||||
|
async fn get_metadata_for_track(&self, track_path: &str) -> Result<Track, ProviderError> {
|
||||||
|
let video = match parse_path(track_path)? {
|
||||||
|
YtPath::SearchTrack { video, .. } | YtPath::PlaylistTrack { video, .. } => video,
|
||||||
|
_ => return Err(ProviderError::MalformedPath),
|
||||||
|
};
|
||||||
|
let entry = self
|
||||||
|
.engine
|
||||||
|
.video_entry(&video_url(video))
|
||||||
|
.await
|
||||||
|
.map_err(|err| engine_err("video metadata", err))?;
|
||||||
|
let parent = crabidy_core::parent_path(track_path).unwrap_or(PROVIDER_ROOT);
|
||||||
|
let mut track = entry_to_track(&entry, parent);
|
||||||
|
// The entry's id names the video; the caller's path is canonical.
|
||||||
|
track.path = track_path.to_string();
|
||||||
|
Ok(track)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `search` always; `playlists` only when logged in.
|
||||||
|
fn get_lib_root(&self) -> LibraryNode {
|
||||||
|
let mut children = vec![LibraryNodeChild {
|
||||||
|
is_creatable: true,
|
||||||
|
..LibraryNodeChild::new(
|
||||||
|
format!("{PROVIDER_ROOT}/search"),
|
||||||
|
"search".to_string(),
|
||||||
|
false,
|
||||||
|
)
|
||||||
|
}];
|
||||||
|
if self.logged_in {
|
||||||
|
children.push(LibraryNodeChild::new(
|
||||||
|
format!("{PROVIDER_ROOT}/playlists"),
|
||||||
|
"playlists".to_string(),
|
||||||
|
false,
|
||||||
|
));
|
||||||
|
}
|
||||||
|
LibraryNode {
|
||||||
|
path: PROVIDER_ROOT.to_string(),
|
||||||
|
title: "youtube".to_string(),
|
||||||
|
parent: Some(crabidy_core::ROOT_PATH.to_string()),
|
||||||
|
tracks: Vec::new(),
|
||||||
|
children,
|
||||||
|
is_queable: false,
|
||||||
|
is_creatable: false,
|
||||||
|
is_downloadable: false,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn get_lib_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
|
||||||
|
let parent = crabidy_core::parent_path(path)
|
||||||
|
.unwrap_or(crabidy_core::ROOT_PATH)
|
||||||
|
.to_string();
|
||||||
|
let node = match parse_path(path)? {
|
||||||
|
YtPath::Root => self.get_lib_root(),
|
||||||
|
YtPath::Search => LibraryNode {
|
||||||
|
path: path.to_string(),
|
||||||
|
title: "search".to_string(),
|
||||||
|
parent: Some(parent),
|
||||||
|
tracks: Vec::new(),
|
||||||
|
children: self
|
||||||
|
.search_terms_snapshot()
|
||||||
|
.iter()
|
||||||
|
.map(|term| {
|
||||||
|
// Term nodes are the modifiable nodes: renamable
|
||||||
|
// (`e`) and deletable (`d`), like tidal's.
|
||||||
|
LibraryNodeChild {
|
||||||
|
is_editable: true,
|
||||||
|
is_deletable: true,
|
||||||
|
..LibraryNodeChild::new(
|
||||||
|
crabidy_core::join_path(path, &crabidy_core::encode_segment(term)),
|
||||||
|
term.clone(),
|
||||||
|
true,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.collect(),
|
||||||
|
is_queable: false,
|
||||||
|
is_creatable: true,
|
||||||
|
is_downloadable: false,
|
||||||
|
},
|
||||||
|
YtPath::SearchTerm(encoded) => {
|
||||||
|
let term = crabidy_core::decode_segment(encoded);
|
||||||
|
// Unknown terms (stale client cache, server restart) are
|
||||||
|
// recreated implicitly instead of erroring.
|
||||||
|
self.register_search_term(&term);
|
||||||
|
self.search_term_node(path, &term, parent).await?
|
||||||
|
}
|
||||||
|
YtPath::Playlists => {
|
||||||
|
if !self.logged_in {
|
||||||
|
warn!(path, "playlists need a configured cookies file");
|
||||||
|
return Err(ProviderError::MalformedPath);
|
||||||
|
}
|
||||||
|
self.playlists_node(path, parent).await?
|
||||||
|
}
|
||||||
|
YtPath::Playlist(playlist_id) => {
|
||||||
|
if !self.logged_in {
|
||||||
|
warn!(path, "playlists need a configured cookies file");
|
||||||
|
return Err(ProviderError::MalformedPath);
|
||||||
|
}
|
||||||
|
self.playlist_node(path, playlist_id, parent).await?
|
||||||
|
}
|
||||||
|
YtPath::SearchTrack { .. } | YtPath::PlaylistTrack { .. } => {
|
||||||
|
warn!(path, "get_lib_node called with a track path");
|
||||||
|
return Err(ProviderError::MalformedPath);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// The central download blessing (architecture/youtube-provider.md
|
||||||
|
// D4, same rule as tidal): every node serving playable content
|
||||||
|
// allows `W`; children mirror their queueability.
|
||||||
|
let mut node = node;
|
||||||
|
node.is_downloadable = node.is_queable || !node.tracks.is_empty();
|
||||||
|
for child in &mut node.children {
|
||||||
|
child.is_downloadable = child.is_queable;
|
||||||
|
}
|
||||||
|
Ok(node)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Only `/youtube/search` is creatable: registers the term and
|
||||||
|
/// returns its node (implicit recreation on stale paths, like
|
||||||
|
/// tidal).
|
||||||
|
async fn create_lib_node(
|
||||||
|
&self,
|
||||||
|
parent_path: &str,
|
||||||
|
title: &str,
|
||||||
|
) -> Result<LibraryNode, ProviderError> {
|
||||||
|
let term = title.trim();
|
||||||
|
if term.is_empty() {
|
||||||
|
return Err(ProviderError::InvalidInput);
|
||||||
|
}
|
||||||
|
if parse_path(parent_path)? != YtPath::Search {
|
||||||
|
warn!(parent_path, "node creation not supported here");
|
||||||
|
return Err(ProviderError::NotSupported);
|
||||||
|
}
|
||||||
|
self.register_search_term(term);
|
||||||
|
let term_path = crabidy_core::join_path(parent_path, &crabidy_core::encode_segment(term));
|
||||||
|
self.get_lib_node(&term_path).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Renaming a search term re-runs the search under the new term.
|
||||||
|
async fn rename_lib_node(
|
||||||
|
&self,
|
||||||
|
path: &str,
|
||||||
|
new_title: &str,
|
||||||
|
) -> Result<LibraryNode, ProviderError> {
|
||||||
|
let YtPath::SearchTerm(encoded) = parse_path(path)? else {
|
||||||
|
warn!(path, "only search terms are renamable");
|
||||||
|
return Err(ProviderError::NotSupported);
|
||||||
|
};
|
||||||
|
let new_term = new_title.trim();
|
||||||
|
if new_term.is_empty() {
|
||||||
|
return Err(ProviderError::InvalidInput);
|
||||||
|
}
|
||||||
|
let old_term = crabidy_core::decode_segment(encoded);
|
||||||
|
// Replace in place; renaming onto an existing term merges (the
|
||||||
|
// duplicate disappears), like tidal's terms.
|
||||||
|
self.remove_search_term(&old_term);
|
||||||
|
self.register_search_term(new_term);
|
||||||
|
let new_path =
|
||||||
|
crabidy_core::join_path("/youtube/search", &crabidy_core::encode_segment(new_term));
|
||||||
|
self.get_lib_node(&new_path).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Deleting a search term is idempotent and returns the refreshed
|
||||||
|
/// search node.
|
||||||
|
async fn delete_lib_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
|
||||||
|
let YtPath::SearchTerm(encoded) = parse_path(path)? else {
|
||||||
|
warn!(path, "only search terms are deletable");
|
||||||
|
return Err(ProviderError::NotSupported);
|
||||||
|
};
|
||||||
|
let term = crabidy_core::decode_segment(encoded);
|
||||||
|
self.remove_search_term(&term);
|
||||||
|
self.get_lib_node("/youtube/search").await
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use std::fs;
|
||||||
|
use std::os::unix::fs::PermissionsExt;
|
||||||
|
use std::path::Path;
|
||||||
|
use tempfile::TempDir;
|
||||||
|
|
||||||
|
/// A fake yt-dlp: a shell script dispatching on its argv. Every test
|
||||||
|
/// engine call goes through it — no network, no real binary.
|
||||||
|
fn fake_binary(dir: &Path, body: &str) -> PathBuf {
|
||||||
|
let path = dir.join("fake-yt-dlp");
|
||||||
|
fs::write(&path, format!("#!/bin/sh\n{body}\n")).expect("write fake binary");
|
||||||
|
let mut perms = fs::metadata(&path).expect("metadata").permissions();
|
||||||
|
perms.set_mode(0o755);
|
||||||
|
fs::set_permissions(&path, perms).expect("chmod");
|
||||||
|
path
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The standard fake: search terms `lofi` (2 results), one playlist
|
||||||
|
/// feed, one playlist, and one resolvable video.
|
||||||
|
const FAKE: &str = r#"
|
||||||
|
case "$*" in
|
||||||
|
*--version*) echo "2026.06.09"; exit 0 ;;
|
||||||
|
*ytsearch2:lofi\ beats*) printf '%s' '{"entries":[{"id":"vidA","title":"Beats","uploader":"Chan"}]}'; exit 0 ;;
|
||||||
|
*ytsearch2:lofi*) printf '%s' '{"entries":[{"id":"vid1","title":"Track One","uploader":"Chan","duration":63.4},{"id":"vid2","title":"Track Two","channel":"Chan Two"}]}'; exit 0 ;;
|
||||||
|
*-g*watch?v=vid1*|*watch?v=vid1*-g*) printf '%s\n%s\n' "https://example.test/a.webm" "https://example.test/b.webm"; exit 0 ;;
|
||||||
|
*watch?v=vid1*) printf '%s' '{"id":"vid1","title":"Track One","uploader":"Chan","duration":63.9}'; exit 0 ;;
|
||||||
|
*feed/playlists*) printf '%s' '{"entries":[{"id":"PL1","title":"Road Mix"}]}'; exit 0 ;;
|
||||||
|
*list=PL1*) printf '%s' '{"title":"Road Mix","entries":[{"id":"vid9","title":"Nine","uploader":"Chan","duration":10}]}'; exit 0 ;;
|
||||||
|
*) echo "unmatched: $*" >&2; exit 1 ;;
|
||||||
|
esac"#;
|
||||||
|
|
||||||
|
async fn client_with(dir: &Path, body: &str, cookies: Option<&Path>) -> Client {
|
||||||
|
let binary = fake_binary(dir, body);
|
||||||
|
let settings = Settings {
|
||||||
|
binary: Some(binary),
|
||||||
|
cookies: cookies.map(Path::to_path_buf),
|
||||||
|
search_results: Some(2),
|
||||||
|
// Generous: under full-workspace parallel test load, process
|
||||||
|
// spawn latency has flaked a 5 s budget.
|
||||||
|
call_timeout_secs: Some(30),
|
||||||
|
};
|
||||||
|
let toml = toml::to_string(&settings).expect("settings toml");
|
||||||
|
Client::init(&toml).await.expect("init")
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn client(dir: &Path) -> Client {
|
||||||
|
client_with(dir, FAKE, None).await
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn init_probes_the_binary() {
|
||||||
|
let dir = TempDir::new().expect("tempdir");
|
||||||
|
// A working probe succeeds…
|
||||||
|
let _ = client(dir.path()).await;
|
||||||
|
// …a missing binary fails init (the orchestrator treats that as
|
||||||
|
// "provider disabled", never as a server error).
|
||||||
|
let settings = Settings {
|
||||||
|
binary: Some(dir.path().join("no-such-binary")),
|
||||||
|
..Settings::default()
|
||||||
|
};
|
||||||
|
let toml = toml::to_string(&settings).expect("settings toml");
|
||||||
|
assert!(Client::init(&toml).await.is_err());
|
||||||
|
// …and so does one that exits non-zero on --version.
|
||||||
|
let broken = TempDir::new().expect("tempdir");
|
||||||
|
fake_binary(broken.path(), "exit 3");
|
||||||
|
let settings = Settings {
|
||||||
|
binary: Some(broken.path().join("fake-yt-dlp")),
|
||||||
|
..Settings::default()
|
||||||
|
};
|
||||||
|
let toml = toml::to_string(&settings).expect("settings toml");
|
||||||
|
assert!(Client::init(&toml).await.is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn root_lists_playlists_only_when_logged_in() {
|
||||||
|
let dir = TempDir::new().expect("tempdir");
|
||||||
|
let anon = client(dir.path()).await;
|
||||||
|
let root = anon.get_lib_root();
|
||||||
|
let titles: Vec<&str> = root.children.iter().map(|c| c.title.as_str()).collect();
|
||||||
|
assert_eq!(titles, vec!["search"]);
|
||||||
|
assert!(root.children[0].is_creatable, "search is creatable");
|
||||||
|
|
||||||
|
let cookies = dir.path().join("cookies.txt");
|
||||||
|
fs::write(&cookies, "# Netscape HTTP Cookie File\n").expect("cookies");
|
||||||
|
let dir2 = TempDir::new().expect("tempdir");
|
||||||
|
let logged_in = client_with(dir2.path(), FAKE, Some(&cookies)).await;
|
||||||
|
let root = logged_in.get_lib_root();
|
||||||
|
let titles: Vec<&str> = root.children.iter().map(|c| c.title.as_str()).collect();
|
||||||
|
assert_eq!(titles, vec!["search", "playlists"]);
|
||||||
|
|
||||||
|
// A configured but unreadable cookies file degrades to logged
|
||||||
|
// out instead of failing init.
|
||||||
|
let dir3 = TempDir::new().expect("tempdir");
|
||||||
|
let missing = dir3.path().join("gone.txt");
|
||||||
|
let degraded = client_with(dir3.path(), FAKE, Some(&missing)).await;
|
||||||
|
assert_eq!(degraded.get_lib_root().children.len(), 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn search_terms_are_created_listed_and_searched() {
|
||||||
|
let dir = TempDir::new().expect("tempdir");
|
||||||
|
let client = client(dir.path()).await;
|
||||||
|
let node = client
|
||||||
|
.create_lib_node("/youtube/search", "lofi")
|
||||||
|
.await
|
||||||
|
.expect("create term");
|
||||||
|
assert_eq!(node.path, "/youtube/search/lofi");
|
||||||
|
assert!(node.is_queable, "pure track results are queueable");
|
||||||
|
assert!(node.is_downloadable, "search results are downloadable");
|
||||||
|
assert_eq!(node.tracks.len(), 2);
|
||||||
|
let one = &node.tracks[0];
|
||||||
|
assert_eq!(one.path, "/youtube/search/lofi/vid1");
|
||||||
|
assert_eq!(one.title, "Track One");
|
||||||
|
assert_eq!(one.artist, "Chan");
|
||||||
|
assert_eq!(one.duration, Some(63));
|
||||||
|
// `channel` is an accepted alias for the artist.
|
||||||
|
assert_eq!(node.tracks[1].artist, "Chan Two");
|
||||||
|
|
||||||
|
// The search node lists the term as an editable/deletable child.
|
||||||
|
let search = client
|
||||||
|
.get_lib_node("/youtube/search")
|
||||||
|
.await
|
||||||
|
.expect("search");
|
||||||
|
assert!(search.is_creatable);
|
||||||
|
assert_eq!(search.children.len(), 1);
|
||||||
|
let child = &search.children[0];
|
||||||
|
assert_eq!(child.title, "lofi");
|
||||||
|
assert!(child.is_editable && child.is_deletable);
|
||||||
|
|
||||||
|
// Encoded terms round trip; unknown terms recreate implicitly.
|
||||||
|
let node = client
|
||||||
|
.get_lib_node("/youtube/search/lofi%20beats")
|
||||||
|
.await
|
||||||
|
.expect("implicit term");
|
||||||
|
assert_eq!(node.tracks.len(), 1);
|
||||||
|
assert_eq!(node.tracks[0].path, "/youtube/search/lofi%20beats/vidA");
|
||||||
|
assert_eq!(client.search_terms_snapshot().len(), 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn search_terms_rename_and_delete() {
|
||||||
|
let dir = TempDir::new().expect("tempdir");
|
||||||
|
let client = client(dir.path()).await;
|
||||||
|
let _ = client
|
||||||
|
.create_lib_node("/youtube/search", "lofi beats")
|
||||||
|
.await
|
||||||
|
.expect("create term");
|
||||||
|
let renamed = client
|
||||||
|
.rename_lib_node("/youtube/search/lofi%20beats", "lofi")
|
||||||
|
.await
|
||||||
|
.expect("rename re-searches");
|
||||||
|
assert_eq!(renamed.path, "/youtube/search/lofi");
|
||||||
|
assert_eq!(renamed.tracks.len(), 2);
|
||||||
|
assert_eq!(client.search_terms_snapshot(), vec!["lofi".to_string()]);
|
||||||
|
|
||||||
|
let search = client
|
||||||
|
.delete_lib_node("/youtube/search/lofi")
|
||||||
|
.await
|
||||||
|
.expect("delete");
|
||||||
|
assert!(search.children.is_empty());
|
||||||
|
// Idempotent: deleting again still returns the search node.
|
||||||
|
let again = client
|
||||||
|
.delete_lib_node("/youtube/search/lofi")
|
||||||
|
.await
|
||||||
|
.expect("idempotent delete");
|
||||||
|
assert!(again.children.is_empty());
|
||||||
|
// Only /youtube/search children are mutable.
|
||||||
|
assert!(client.rename_lib_node("/youtube", "nope").await.is_err());
|
||||||
|
assert!(client.delete_lib_node("/youtube/playlists").await.is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn playlists_list_and_resolve_when_logged_in() {
|
||||||
|
let dir = TempDir::new().expect("tempdir");
|
||||||
|
let cookies = dir.path().join("cookies.txt");
|
||||||
|
fs::write(&cookies, "# cookies\n").expect("cookies");
|
||||||
|
let client = client_with(dir.path(), FAKE, Some(&cookies)).await;
|
||||||
|
|
||||||
|
let playlists = client
|
||||||
|
.get_lib_node("/youtube/playlists")
|
||||||
|
.await
|
||||||
|
.expect("playlists feed");
|
||||||
|
assert_eq!(playlists.children.len(), 1);
|
||||||
|
let pl = &playlists.children[0];
|
||||||
|
assert_eq!(pl.path, "/youtube/playlists/PL1");
|
||||||
|
assert_eq!(pl.title, "Road Mix");
|
||||||
|
assert!(pl.is_queable && pl.is_downloadable);
|
||||||
|
|
||||||
|
let node = client.get_lib_node(&pl.path).await.expect("playlist");
|
||||||
|
assert_eq!(node.title, "Road Mix");
|
||||||
|
assert!(node.is_queable && node.is_downloadable);
|
||||||
|
assert_eq!(node.tracks.len(), 1);
|
||||||
|
assert_eq!(node.tracks[0].path, "/youtube/playlists/PL1/vid9");
|
||||||
|
|
||||||
|
// Logged out, the playlists subtree is a malformed path.
|
||||||
|
let dir2 = TempDir::new().expect("tempdir");
|
||||||
|
let anon = client_with(dir2.path(), FAKE, None).await;
|
||||||
|
assert!(anon.get_lib_node("/youtube/playlists").await.is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn tracks_resolve_streams_and_metadata() {
|
||||||
|
let dir = TempDir::new().expect("tempdir");
|
||||||
|
let client = client(dir.path()).await;
|
||||||
|
assert!(client.is_track_path("/youtube/search/lofi/vid1"));
|
||||||
|
assert!(client.is_track_path("/youtube/playlists/PL1/vid9"));
|
||||||
|
assert!(!client.is_track_path("/youtube/search/lofi"));
|
||||||
|
|
||||||
|
let urls = client
|
||||||
|
.get_urls_for_track("/youtube/search/lofi/vid1")
|
||||||
|
.await
|
||||||
|
.expect("stream urls");
|
||||||
|
assert_eq!(
|
||||||
|
urls,
|
||||||
|
vec![
|
||||||
|
"https://example.test/a.webm".to_string(),
|
||||||
|
"https://example.test/b.webm".into()
|
||||||
|
]
|
||||||
|
);
|
||||||
|
let track = client
|
||||||
|
.get_metadata_for_track("/youtube/playlists/PL1/vid1")
|
||||||
|
.await
|
||||||
|
.expect("metadata");
|
||||||
|
assert_eq!(track.title, "Track One");
|
||||||
|
assert_eq!(track.artist, "Chan");
|
||||||
|
assert_eq!(track.duration, Some(63));
|
||||||
|
assert_eq!(track.path, "/youtube/playlists/PL1/vid1");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn engine_failures_are_typed_never_panics() {
|
||||||
|
let dir = TempDir::new().expect("tempdir");
|
||||||
|
// Non-zero exit on everything but the probe.
|
||||||
|
let body = r#"
|
||||||
|
case "$*" in
|
||||||
|
*--version*) echo 1.0; exit 0 ;;
|
||||||
|
*) echo boom >&2; exit 1 ;;
|
||||||
|
esac"#;
|
||||||
|
let client = client_with(dir.path(), body, None).await;
|
||||||
|
assert!(client.get_lib_node("/youtube/search/lofi").await.is_err());
|
||||||
|
assert!(client
|
||||||
|
.get_urls_for_track("/youtube/search/lofi/vid1")
|
||||||
|
.await
|
||||||
|
.is_err());
|
||||||
|
|
||||||
|
// Malformed JSON is a typed error, not a panic.
|
||||||
|
let dir2 = TempDir::new().expect("tempdir");
|
||||||
|
let body = r#"
|
||||||
|
case "$*" in
|
||||||
|
*--version*) echo 1.0; exit 0 ;;
|
||||||
|
*) printf 'not json'; exit 0 ;;
|
||||||
|
esac"#;
|
||||||
|
let client = client_with(dir2.path(), body, None).await;
|
||||||
|
assert!(client.get_lib_node("/youtube/search/lofi").await.is_err());
|
||||||
|
|
||||||
|
// A hung binary hits the per-call timeout.
|
||||||
|
let dir3 = TempDir::new().expect("tempdir");
|
||||||
|
let body = r#"
|
||||||
|
case "$*" in
|
||||||
|
*--version*) echo 1.0; exit 0 ;;
|
||||||
|
*) sleep 30 ;;
|
||||||
|
esac"#;
|
||||||
|
let binary = fake_binary(dir3.path(), body);
|
||||||
|
let settings = Settings {
|
||||||
|
binary: Some(binary),
|
||||||
|
call_timeout_secs: Some(1),
|
||||||
|
search_results: Some(2),
|
||||||
|
..Settings::default()
|
||||||
|
};
|
||||||
|
let toml = toml::to_string(&settings).expect("settings toml");
|
||||||
|
let client = Client::init(&toml).await.expect("init");
|
||||||
|
let started = std::time::Instant::now();
|
||||||
|
assert!(client.get_lib_node("/youtube/search/lofi").await.is_err());
|
||||||
|
assert!(
|
||||||
|
started.elapsed() < Duration::from_secs(10),
|
||||||
|
"timed out late"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn foreign_and_malformed_paths_are_rejected() {
|
||||||
|
let dir = TempDir::new().expect("tempdir");
|
||||||
|
let client = client(dir.path()).await;
|
||||||
|
for path in ["/tidal/artists", "/youtube/nope", "/youtube/search/a/b/c"] {
|
||||||
|
assert!(client.get_lib_node(path).await.is_err(), "{path}");
|
||||||
|
}
|
||||||
|
assert!(client.create_lib_node("/youtube", "term").await.is_err());
|
||||||
|
}
|
||||||
|
}
|
||||||
Loading…
Reference in New Issue