# YouTube provider on rustypipe (pure Rust)

## Context and problem statement

The first `ytdy` iteration shelled out to `yt-dlp` (a Python tool). Two
problems surfaced in real use:

1. **Playback was broken.** Search worked, but picked tracks never
   played: `yt-dlp -f bestaudio` selects WebM/**Opus**, and the player is
   rodio + symphonia (`symphonia-all`) — symphonia has **no Opus
   decoder**. The stream downloaded fine and then failed to decode.
   (Download captures of YouTube tracks had the same latent problem:
   `.webm` files that the local player cannot decode.)
2. The user does not want a Python subprocess in the loop.

## Evaluation of pure-Rust extractors (2026-07-21, live-tested)

- `rusty_ytdl` 0.7.4 — search works, but **stream URLs come back
  empty** and `stream()` fails with "Video source empty": its cipher
  handling has fallen behind YouTube's rotation (last release early
  2025).
- `rustube` 0.6.0 — unmaintained since ~2022; not tested further.
- `rust-yt-downloader` 0.1.0 — a small CLI, not a library engine.
- `rustypipe` 0.11.4 — **works end to end**: search, video details,
  playlists, and `player()` returns deciphered stream URLs (verified:
  HTTP 206 fetch, and the full itag-140 m4a of a test video decodes
  through `rodio::Decoder` — 44.1 kHz samples out). Actively maintained
  (NewPipe-inspired Innertube client).

**Decision: replace the subprocess engine with `rustypipe`.** It removes
the Python dependency *and* the format problem: we pick the stream
ourselves and prefer `audio/mp4` (AAC — symphonia decodes it) over
Opus. Captures get `.m4a` audio the local player can play.

Robustness note: any non-yt-dlp extractor can break when YouTube
changes Innertube. rustypipe is the most actively maintained Rust
option, ships `rustypipe-botguard` (an optional *Rust* helper binary,
auto-detected on PATH) for PO-token attestation if YouTube starts
demanding it, and persists client state in a cache file. Accepted risk,
revisit if streams start failing.

## D1 — Extractor seam

The subprocess `Engine` is replaced by an `Extractor` trait owned by
`ytdy` (search videos, video details, audio stream URL, saved
playlists, playlist videos) with two implementations:

- `RustyPipeExtractor` — the real one, wrapping one `RustyPipe` client.
- a test fake — provider logic (path scheme, search-term store, node
  shapes, gating) is tested without network or fake shell scripts.

`Client` keeps its public `ProviderClient` surface, path scheme, and
the in-memory search-term store unchanged.

```d2
direction: right
tui -> server -> ytdy: "/youtube/..."
ytdy: {
  client: "Client\n(paths, search terms, nodes)"
  extractor: "Extractor trait"
  client -> extractor
}
ytdy.extractor -> rustypipe: "RustyPipeExtractor"
rustypipe -> youtube: "Innertube (HTTPS)"
tests -> ytdy.extractor: "FakeExtractor"
```

## D2 — Stream selection (the playback fix)

`audio_stream_url` prefers the highest-average-bitrate `audio/mp4`
stream (AAC — decodable by the player); only if none exists does it
fall back to the overall best audio stream, with a warning (it will
likely not decode locally, but the URL is still honest — e.g. a future
player may cope). Download captures inherit the same choice, so their
audio is `.m4a`.

## D3 — Login and playlists

The `cookies` setting keeps its meaning: a path to a Netscape
`cookies.txt` export. rustypipe consumes it natively
(`user_auth_set_cookie_txt`) and — importantly — **persists the rotated
cookie in its cache file**, which outlives the (quickly stale) original
export. Init order: if the cache already holds a working login
(`user_auth_check_cookie`), use it; otherwise load the configured file;
on any failure degrade to logged-out with a warning (never a failed
init). "Logged in" gates the `playlists` subtree exactly as before,
now backed by `saved_playlists()` (the `userdata` feature) instead of a
live-unvalidated `feed/playlists` scrape.

Playlist nodes page through `Paginator::extend_limit` up to a track
cap (`MAX_PLAYLIST_TRACKS`, 1000) instead of loading whole playlists
blindly.

## D4 — Configuration and environment

`ytdy.toml`: `binary` is gone (nothing to spawn); `cookies` and
`search_results` stay; `call_timeout_secs` maps to the rustypipe
client timeout. New `RustyPipe` client state lives in
`<config>/crabidy/rustypipe/` (`storage_dir`) — it holds the rotated
auth cookie, so it is as secret as the cookies file; neither its
contents nor cookie values are ever logged. `yt-dlp` leaves
`devenv.nix`. Init never probes the network except when validating a
configured login; a failed login check degrades, a broken client build
disables the provider non-fatally (as before).

## D5 — Out of scope

- Installing `rustypipe-botguard` (optional PO-token helper); document
  only. Streams work without it today.
- Opus support in the player (a symphonia Opus decoder does not exist;
  an opus feature via a different rodio decoder is a separate project).
- YouTube Music (rustypipe supports it; nothing in crabidy asks yet).

## Risks

- Innertube changes can break rustypipe between releases; mitigations:
  cache-backed client data, optional botguard, active upstream.
- `saved_playlists` needs valid cookies; YouTube rotates them — the
  cache keeps the rotated value, but a long-cold server may need a
  fresh export. Degrades to logged-out, never fails.