clusterfuck
/
crabidy
4
1
Fork 0
Code Issues 15 Pull Requests Actions Packages Projects 1 Releases 5 Wiki Activity
crabidy/architecture/fyyd-provider.md

9.7 KiB
Raw Blame History

fyyd provider (podcasts)

Context and problem statement

A new library provider mounted at /fyyd that lets a user find and play podcasts, backed by fyyd's public search engine.

  • Search works with no account — creatable search-term nodes exactly like /tidal/search and /youtube/search (% creates a term, results appear underneath).
  • Unlike YouTube, a podcast search does not return tracks directly: it returns podcasts, each of which is a container of episodes. So the tree carries one extra level — search-term → podcast → episodes — where YouTube is search-term → tracks.
  • Playing an episode means streaming its enclosure URL (the plain HTTP(S) audio file from the podcast's RSS feed). That is exactly the URL-returning shape the audio player already handles; no sidecar, no cipher solving, no byte-proxying (contrast /youtube).
  • Captures (W, download) come for free: any node that serves tracks and raises is_downloadable gets w/W with no wire or TUI work.

The fyyd API (grounding)

Base https://api.fyyd.de/0.2/, no key or auth for search and browse. Responses are wrapped in a JSON envelope { "status", "msg", "data", … }; list endpoints add meta.paging. The endpoints we use:

Purpose Endpoint
Search podcasts GET /search/podcast?term=<t>&count=<n>
Podcast episodes GET /podcast/episodes?podcast_id=<id>&count=<n>
Hot (featured) podcasts GET /feature/podcast/hot?count=<n>
Single episode GET /episode?episode_id=<id>

Objects (fields we read):

  • podcast: id (int), title, xmlURL, imgURL, description, language.
  • episode: id (int), title, guid, enclosure (audio URL), podcast_id, duration (seconds), pubdate.

The episode's enclosure is the only field playback needs. The single episode endpoint does not carry the podcast's title, so an episode fetched on its own has no "artist" until we look the podcast up (see D4).

Assumptions (decided here)

  • The captures/creatable/editable/deletable TUI flows are provider-agnostic (confirmed by /youtube): mirroring tidal's search-term semantics costs no TUI or wire change. No proto change, no new ProviderCommand.
  • fyyd's public API needs no credentials, so — unlike tidal — a missing or empty fyyd.toml is the normal case, and init never needs a login. The provider is non-fatal at startup like the local providers: a build/parse failure only costs the /fyyd subtree.
  • The audio player streams plain https enclosure URLs directly (audio-player windowed-HTTP path). Podcast enclosures are ordinary media files, so no /youtube-style URL-lifetime or ~1 MiB-cap problem.
  • Episodes are addressed by fyyd's stable numeric id (as a string segment). We do not use the RSS guid as the path key — the fyyd id is shorter, already URL-safe, and is what the episode endpoint takes.

Decisions

D1 — Crate fyyd, mounted at /fyyd, non-fatal init

New workspace crate fyyd implementing ProviderClient, shaped on ytdy (the closest analog: remote, search-driven, in-memory terms). Wired into ProviderOrchestrator with a fyyd_client: Option<Arc<fyyd::Client>> field, fyyd_owns()/fyyd_provider() helpers, a build() block that reads fyyd.toml (non-fatal), a get_lib_root child gated on self.fyyd_client.is_some(), and one routing arm in each dispatch method. crabidy-server's settings gain fyyd in ALL_PROVIDERS, in ProviderToggles, in all(), and in provider_toggles().

D2 — HTTP behind a trait, faked in tests

All network access goes through one seam — a Fyyd trait (search_podcasts, hot_podcasts, podcast_episodes, episode) behind Box<dyn Fyyd> — with a reqwest-based FyydApi for production and a FakeApi in tests, exactly as ytdy hides rustypipe behind Extract. Provider logic (tree shaping, path parsing, term store) is then unit-tested with zero network. The trait's error type maps to ProviderError::FetchError at the boundary; malformed paths are MalformedPath; empty create/rename input is InvalidInput.

D3 — Tree shape (the extra level)

  • /fyyd — children: search (always, is_creatable), hot (always, a fixed featured-podcasts browse so the provider is useful with zero typing).
  • /fyyd/search — is_creatable; children are the in-memory search terms (RwLock<Vec<String>>, dedup, recreated implicitly on stale paths), each is_editable + is_deletable, like tidal/youtube.
  • /fyyd/search/<term> — lists the top-N matching podcasts as children (containers, not tracks). Not directly queueable itself; each podcast child is queueable.
  • /fyyd/search/<term>/<podcast-id> — lists that podcast's episodes as tracks; queueable and downloadable (homogeneous tracks). Queueing the podcast node enqueues its listed episodes (bounded, see D5).
  • /fyyd/search/<term>/<podcast-id>/<episode-id> — the track leaf.
  • /fyyd/hot — lists featured podcasts as children (same podcast shape as a search-term node, but the list comes from /feature/podcast/hot instead of a term).
  • /fyyd/hot/<podcast-id> and /fyyd/hot/<podcast-id>/<episode-id> — identical podcast/episode shapes as under search; the two branches share the podcast-node and episode-leaf builders.

Terms are percent-encoded into one segment (encode_segment/ decode_segment); podcast and episode ids are already URL-safe integers.

D4 — Streams, metadata, artist

  • get_urls_for_track: parse the episode id from the path → episode(id) → vec![enclosure]. An empty/missing enclosure is FetchError, not a panic; the queue skips it.
  • Track fields: title = episode title, artist = the podcast title, duration = episode duration (seconds → Option<u32>), provider_item_id = fyyd episode id (keys the content store for captures), album = None.
  • Artist source. When episodes are listed under a podcast we already hold the podcast title, so listed tracks get the right artist for free. A directly fetched episode (get_metadata_for_track on a bare episode path, or a stream resolve) does not carry the podcast title from the episode endpoint; the FyydApi fills it with one extra /podcast lookup, and the Episode model carries artist: Option<String> so the fake can supply it in tests. A missing artist degrades to an empty string, never an error.

D5 — Bounds and freshness

  • search_results (podcasts per term, default 20), hot_count (default 20), and episodes_per_podcast (default 100) are configurable in fyyd.toml; every listing is capped so a huge podcast cannot stall the library or queue resolution. A call_timeout_secs (default 30) bounds each HTTP call (hard rule: timeouts on external calls).
  • Listings are fetched fresh per call (no cross-call cache), like tidal's and youtube's fresh remote calls. Only the search terms are stored, in memory.

D6 — Out of scope (explicitly)

  • fyyd user accounts, OAuth, subscriptions, personal collections, or the "hot languages" / category browse — search + hot cover the "find and play" ask.
  • Episode chapters, transcripts, or per-episode images in the queue.
  • Pagination past the configured caps (one page is fetched per listing).
  • Caching or persisting episodes to disk beyond the existing W captures.

Structure

direction: right

server: crabidy-server {
  orch: ProviderOrchestrator
}

fyyd: "fyyd (crate)" {
  client: "Client\n(ProviderClient)"
  terms: "search terms\n(in-memory, like tidal)"
  api: "FyydApi\n(reqwest seam: Fyyd trait)"
  client -> terms
  client -> api
}

svc: "api.fyyd.de\n(public, keyless)" { shape: cloud }
cdn: "podcast enclosure\n(RSS media host)" { shape: cloud }
player: audio-player { shape: hexagon }

server.orch -> fyyd.client: "/fyyd/..."
fyyd.api -> svc: "search / episodes / hot (JSON, timeout)"
server.orch -> player: "enclosure URL"
player -> cdn: "windowed HTTP stream"

Key flow: search a podcast, play an episode

shape: sequence_diagram
tui: TUI
orch: Orchestrator
f: fyyd
api: api.fyyd.de

tui -> orch: "% on /fyyd/search: 'history'"
orch -> f: "create_lib_node"
f -> api: "GET /search/podcast?term=history"
api -> f: "podcasts (id, title)"
f -> tui: "term node: podcasts as children"
tui -> orch: "open a podcast"
orch -> f: "get_lib_node(/fyyd/search/history/<pid>)"
f -> api: "GET /podcast/episodes?podcast_id=<pid>"
api -> f: "episodes (id, title, enclosure, duration)"
f -> tui: "podcast node: episodes as tracks (queueable)"
tui -> orch: "queue + play an episode"
orch -> f: "get_urls_for_track(.../<eid>)"
f -> api: "GET /episode?episode_id=<eid>"
api -> f: "enclosure URL"
orch -> orch: "player streams the enclosure"

Risks and open questions

  • fyyd envelope / field drift. The data envelope and field names (enclosure, xmlURL, numeric id) are read from the current public docs but were not live-validated during design. The FyydApi decodes defensively (serde with #[serde(default)], missing fields degrade, no panic) and every failure is a typed FetchError; if a field name is wrong the fix is local to FyydApi's DTOs. Live validation is a task-plan gate.
  • Episode enclosure availability / expiry. Some feeds proxy or expire enclosures; a dead URL surfaces as a skipped track, never a crash.
  • Artist double-fetch. Filling a directly-fetched episode's artist costs one extra /podcast call; acceptable because direct metadata fetches are rare (listing is the common path and already has the title).
  • Rate limits. fyyd does not document limits; the per-call timeout and the absence of background polling keep request volume to user actions.