clusterfuck
/
crabidy
4
1
Fork 0
Code Issues 15 Pull Requests Actions Packages Projects 1 Releases 5 Wiki Activity
crabidy/architecture/fs-provider.md

10 KiB
Raw Blame History

Filesystem provider

Context and problem statement

crabidy currently has exactly one media provider (Tidal, crate tidaldy) behind the ProviderClient trait and the ProviderOrchestrator that routes by path prefix. The user wants a second provider that walks a local directory tree and treats files with a well-known extension as serialized track nodes: small metadata files that describe a track and point at the thing that actually plays. The playable reference can be

  1. a local audio file (mp3/flac/… somewhere on disk),
  2. a web URL (a stream, a radio station, a direct http(s) link), or
  3. a crabidy-internal link (a track path owned by another provider, e.g. /tidal/artists/3634161/536243361).

The request's open question — "new datastructure or our existing node?" — is decided below (D1: existing node on the wire, a new on-disk schema for the file).

Assumptions (confirmed against the code)

  • audio-player already plays both cases we need natively (player_engine.rs): a source string that parses as an http(s) URL is streamed via stream-download; anything else is opened as a local file path. No player changes are required. (file:// URLs would be rejected — the provider must return plain paths, not file URLs.)
  • Track.path is the routing key for playback: the queue stores whole Track messages, and GetTrackUrls/get_metadata_for_track route by the path's first segment in ProviderOrchestrator. Nothing in the server assumes a track's path belongs to the provider whose node listed it.
  • The default ProviderClient::resolve_tracks_into walk (one chunk per track-bearing node, pre-order) is fast enough for local disk I/O; the page-streaming override exists for slow remote APIs.
  • The TUI needs no changes: /fs appears as one more child of the synthetic root, directories are nodes, track files are tracks.

Decisions

D1 — Reuse Track/LibraryNode; the only new schema is on disk

Options considered:

  • (a) New proto message (e.g. TrackRef with a oneof playable) carried through queue, RPCs, and TUI.
  • (b) Reuse the existing Track/LibraryNode messages unchanged; the "reference to a playable thing" lives only inside the fs provider's on-disk file and is resolved to ordinary crabidy semantics at the provider boundary.

Decision: (b). A new wire type would ripple through the queue, every RPC, and both clients for zero client-visible benefit — the queue and TUI only ever need metadata + a playable path, which Track already is. The new datastructure is purely the serialized track-file schema (D3), private to the fs provider crate.

D2 — Internal links resolve by path rewriting at listing time

Options considered:

  • (a) Keep Track.path = /fs/... for link tracks and add an indirection mechanism at play time (orchestrator re-dispatches get_urls_for_track when the fs provider reports a redirect).
  • (b) When the fs provider builds a Track from a link file, it sets Track.path to the link target (e.g. /tidal/...). The file's own metadata still fills artist/title/album. From then on the track is a tidal track as far as the queue and playback are concerned; the orchestrator's existing prefix routing does the rest.

Decision: (b). Zero new mechanisms: get_urls_for_track and metadata refresh route to the owning provider automatically, and a dead target degrades exactly like any other dead tidal track (playback warn + skip). Consequences, accepted deliberately:

  • The queue shows the metadata written in the file (authoritative by the user's own description), not the target's live metadata.
  • get_urls_for_track on an fs path whose playable is a link cannot occur through normal flow (the path was rewritten before it could be queued); if it happens anyway it is MalformedPath with a warning, not a chain resolution. Link chains are structurally impossible — see D3's "no links into /fs" rule.

D3 — On-disk schema: TOML, extension .track.toml, exactly one playable

TOML per project convention. A file named <anything>.track.toml inside the configured root is a track node; everything else (other files, hidden entries) is ignored. Schema:

# Required.
title = "We Will Rock You"
# Optional; empty when omitted (web radio streams often have no artist).
artist = "Queen"
# Optional, seconds.
duration = 122

# Optional.
[album]
title = "News of the World"
release_date = "1977-10-28"

# Required: exactly one of `file`, `url`, `link`.
[playable]
file = "../flac/we-will-rock-you.flac"
# url = "https://example.org/stream.mp3"
# link = "/tidal/artists/3634161/536243361"
  • playable is parsed as a struct of three Options and validated to exactly one set — this gives precise error messages, unlike an untagged serde enum.
  • file: absolute, or relative to the track file's directory (so a music folder stays relocatable). Existence is not checked at listing time (TOCTOU; the player produces a good error at play time).
  • url: must parse as http/https (matching what the player accepts).
  • link: must be an absolute crabidy path (/-prefixed) and must not point into /fs — self-links would allow chains/cycles; other providers are one hop away by construction.
  • A file that fails to parse or validate is skipped with a warning at listing time; it never panics and never poisons its directory (hard rule: no panic on user input).

D4 — Library mapping: one configured root, encoded segments, sorted listing

  • Config ~/.config/crabidy/fsdy.toml, written back with defaults on first run like tidaly.toml. Single field root (absolute path); default dirs::audio_dir() (~/Music). One root keeps the path scheme flat; multiple roots stay future work (they would need a /fs/<root-name>/ layer).
  • Paths: /fs/<seg>/<seg>/… where each segment is encode_segment(file_name) — the same escaping search terms use, so arbitrary file names (spaces, %, unicode) survive the path scheme.
  • Traversal safety: decoded segments are rejected if they are ./.. or contain a path separator; the joined path is a pure descent from the root by construction.
  • Symlinks are skipped during directory listing (file_type() without follow) — no cycles, no escaping the root. A playable.file target may be a symlink; that is the player's problem.
  • Listing order: directories and track files each sorted case-insensitively by file name — deterministic queueing order; users order albums with 01-style file name prefixes as everywhere else.
  • Directories are LibraryNodeChild { is_queable: true }; queueing one resolves its whole subtree via the default resolve_tracks_into walk (one chunk per directory — local disk needs no page streaming). Empty directories are fine: they contribute nothing.
  • Fresh read on every navigation, no cache, no file watching — edits with a file manager appear on the next visit.

D5 — New crate fsdy, non-fatal init, orchestrator routing

  • New workspace crate fsdy (naming symmetry with tidaldy), PROVIDER_ROOT = "/fs", implementing ProviderClient.
  • ProviderOrchestrator gains fs_client: Option<Arc<fsdy::Client>> and routes /fs prefixes in every trait method; get_lib_root adds the /fs child only when the client exists. Init failure is non-fatal (warn + run without /fs): unlike Tidal, a broken local config must not take the whole server down, and existing installations have no fsdy.toml yet. All I/O through tokio::fs (no blocking the runtime).

D6 — Out of scope (explicitly)

  • create/rename/delete_lib_node: NotSupported. Track files are edited with normal file tools; a TUI editor for them is future work.
  • Reading audio-file tags (ID3 etc.) to synthesize track nodes for plain .mp3 files sitting in the tree: future work — this feature is about the serialized-node format.
  • Multiple roots, file watching, link chains: rejected above.

Structure

direction: right

disk: Local disk {
  shape: cylinder
  tree: "root dir: dirs, *.track.toml"
}

server: crabidy-server {
  playback: Playback loop
  orch: ProviderOrchestrator {
    route: "route by first path segment"
  }
}

fsdy: fsdy::Client {
  parse: "parse + validate .track.toml"
  map: "path <-> root-relative file (encoded segments)"
}

tidaldy: tidaldy::Client

player: audio-player {
  url: "http(s) -> stream-download"
  file: "other -> File::open"
}

server.playback -> server.orch: "GetTrackUrls(track.path)"
server.orch -> fsdy: "/fs/..."
server.orch -> tidaldy: "/tidal/..."
fsdy -> disk.tree: tokio::fs
server.playback -> player: "play(url | file path)"

Key flow: queue a directory containing all three playable kinds

shape: sequence_diagram
tui: TUI
pb: Playback loop
orch: Orchestrator
fs: fsdy
tidal: tidaldy

tui -> pb: "ReplaceQueue([/fs/mix])"
pb -> orch: ResolveTracks("/fs/mix", chunk_tx)
orch -> fs: resolve_tracks_into (spawned)
fs -> fs: "list dir, parse 3 track files"
fs -> pb: "chunk of 3 Tracks (paths below)" {style.bold: true}
pb -> orch: "GetTrackUrls(/fs/mix/a.track.toml)"
orch -> fs: get_urls_for_track
fs -> pb: "[/home/u/Music/a.flac]"
pb -> orch: "GetTrackUrls(/tidal/...)  # link track, rewritten path"
orch -> tidal: get_urls_for_track
tidal -> pb: "[https://tidal-cdn/...]"

(The second track's Track.path stays /fs/... — its playable is a URL, returned by fsdy::get_urls_for_track. Only link files rewrite the path.)

Risks and open questions

  • Malicious/odd trees: deep nesting is bounded only by the walk's worklist (memory-cheap); huge directories list in one node — accepted for local disk. Traversal and symlink escapes are closed by D4.
  • Dangling references: dead file/url/link targets surface at play time as the existing "failed to open / no provider owns" warnings; the queue keeps going. No preflight validation by design.
  • Metadata drift on link tracks (file says X, target now titled Y): accepted; the file is the user's curated metadata.
  • Open (future): tag-reading for bare audio files; multiple roots; a %-style creator that writes a .track.toml from inside the TUI.