245 lines
10 KiB
Markdown
245 lines
10 KiB
Markdown
# Filesystem provider
|
|
|
|
## Context and problem statement
|
|
|
|
crabidy currently has exactly one media provider (Tidal, crate `tidaldy`)
|
|
behind the `ProviderClient` trait and the `ProviderOrchestrator` that routes
|
|
by path prefix. The user wants a second provider that walks a local
|
|
directory tree and treats files with a well-known extension as *serialized
|
|
track nodes*: small metadata files that describe a track and point at the
|
|
thing that actually plays. The playable reference can be
|
|
|
|
1. a **local audio file** (mp3/flac/… somewhere on disk),
|
|
2. a **web URL** (a stream, a radio station, a direct http(s) link), or
|
|
3. a **crabidy-internal link** (a track path owned by another provider,
|
|
e.g. `/tidal/artists/3634161/536243361`).
|
|
|
|
The request's open question — "new datastructure or our existing node?" —
|
|
is decided below (D1: existing node on the wire, a new on-disk schema for
|
|
the file).
|
|
|
|
## Assumptions (confirmed against the code)
|
|
|
|
- `audio-player` already plays both cases we need natively
|
|
(`player_engine.rs`): a source string that parses as an `http(s)` URL is
|
|
streamed via `stream-download`; anything else is opened as a **local
|
|
file path**. No player changes are required. (`file://` URLs would be
|
|
rejected — the provider must return plain paths, not file URLs.)
|
|
- `Track.path` is the routing key for playback: the queue stores whole
|
|
`Track` messages, and `GetTrackUrls`/`get_metadata_for_track` route by
|
|
the path's first segment in `ProviderOrchestrator`. Nothing in the
|
|
server assumes a track's path belongs to the provider whose node listed
|
|
it.
|
|
- The default `ProviderClient::resolve_tracks_into` walk (one chunk per
|
|
track-bearing node, pre-order) is fast enough for local disk I/O; the
|
|
page-streaming override exists for slow remote APIs.
|
|
- The TUI needs **no changes**: `/fs` appears as one more child of the
|
|
synthetic root, directories are nodes, track files are tracks.
|
|
|
|
## Decisions
|
|
|
|
### D1 — Reuse `Track`/`LibraryNode`; the only new schema is on disk
|
|
|
|
Options considered:
|
|
|
|
- *(a)* New proto message (e.g. `TrackRef` with a `oneof playable`) carried
|
|
through queue, RPCs, and TUI.
|
|
- *(b)* Reuse the existing `Track`/`LibraryNode` messages unchanged; the
|
|
"reference to a playable thing" lives only inside the fs provider's
|
|
on-disk file and is resolved to ordinary crabidy semantics at the
|
|
provider boundary.
|
|
|
|
**Decision: (b).** A new wire type would ripple through the queue, every
|
|
RPC, and both clients for zero client-visible benefit — the queue and TUI
|
|
only ever need *metadata + a playable path*, which `Track` already is. The
|
|
new datastructure is purely the **serialized track-file schema** (D3),
|
|
private to the fs provider crate.
|
|
|
|
### D2 — Internal links resolve by *path rewriting* at listing time
|
|
|
|
Options considered:
|
|
|
|
- *(a)* Keep `Track.path = /fs/...` for link tracks and add an indirection
|
|
mechanism at play time (orchestrator re-dispatches `get_urls_for_track`
|
|
when the fs provider reports a redirect).
|
|
- *(b)* When the fs provider builds a `Track` from a link file, it sets
|
|
`Track.path` to the **link target** (e.g. `/tidal/...`). The file's own
|
|
metadata still fills artist/title/album. From then on the track *is* a
|
|
tidal track as far as the queue and playback are concerned; the
|
|
orchestrator's existing prefix routing does the rest.
|
|
|
|
**Decision: (b).** Zero new mechanisms: `get_urls_for_track` and metadata
|
|
refresh route to the owning provider automatically, and a dead target
|
|
degrades exactly like any other dead tidal track (playback warn + skip).
|
|
Consequences, accepted deliberately:
|
|
|
|
- The queue shows the metadata written in the file (authoritative by the
|
|
user's own description), not the target's live metadata.
|
|
- `get_urls_for_track` on an fs path whose playable is a link cannot occur
|
|
through normal flow (the path was rewritten before it could be queued);
|
|
if it happens anyway it is `MalformedPath` with a warning, not a chain
|
|
resolution. **Link chains are structurally impossible** — see D3's "no
|
|
links into `/fs`" rule.
|
|
|
|
### D3 — On-disk schema: TOML, extension `.track.toml`, exactly one playable
|
|
|
|
TOML per project convention. A file named `<anything>.track.toml` inside
|
|
the configured root is a track node; everything else (other files, hidden
|
|
entries) is ignored. Schema:
|
|
|
|
```toml
|
|
# Required.
|
|
title = "We Will Rock You"
|
|
# Optional; empty when omitted (web radio streams often have no artist).
|
|
artist = "Queen"
|
|
# Optional, seconds.
|
|
duration = 122
|
|
|
|
# Optional.
|
|
[album]
|
|
title = "News of the World"
|
|
release_date = "1977-10-28"
|
|
|
|
# Required: exactly one of `file`, `url`, `link`.
|
|
[playable]
|
|
file = "../flac/we-will-rock-you.flac"
|
|
# url = "https://example.org/stream.mp3"
|
|
# link = "/tidal/artists/3634161/536243361"
|
|
```
|
|
|
|
- `playable` is parsed as a struct of three `Option`s and validated to
|
|
**exactly one** set — this gives precise error messages, unlike an
|
|
untagged serde enum.
|
|
- `file`: absolute, or relative to the *track file's directory* (so a
|
|
music folder stays relocatable). Existence is **not** checked at listing
|
|
time (TOCTOU; the player produces a good error at play time).
|
|
- `url`: must parse as `http`/`https` (matching what the player accepts).
|
|
- `link`: must be an absolute crabidy path (`/`-prefixed) and must **not**
|
|
point into `/fs` — self-links would allow chains/cycles; other providers
|
|
are one hop away by construction.
|
|
- A file that fails to parse or validate is **skipped with a warning** at
|
|
listing time; it never panics and never poisons its directory (hard
|
|
rule: no panic on user input).
|
|
|
|
### D4 — Library mapping: one configured root, encoded segments, sorted listing
|
|
|
|
- Config `~/.config/crabidy/fsdy.toml`, written back with defaults on
|
|
first run like `tidaly.toml`. Single field `root` (absolute path);
|
|
default `dirs::audio_dir()` (`~/Music`). One root keeps the path scheme
|
|
flat; multiple roots stay future work (they would need a
|
|
`/fs/<root-name>/` layer).
|
|
- Paths: `/fs/<seg>/<seg>/…` where each segment is
|
|
`encode_segment(file_name)` — the same escaping search terms use, so
|
|
arbitrary file names (spaces, `%`, unicode) survive the path scheme.
|
|
- **Traversal safety**: decoded segments are rejected if they are `.`/`..`
|
|
or contain a path separator; the joined path is a pure descent from the
|
|
root by construction.
|
|
- **Symlinks are skipped** during directory listing (`file_type()` without
|
|
follow) — no cycles, no escaping the root. A `playable.file` target may
|
|
be a symlink; that is the player's problem.
|
|
- Listing order: directories and track files each sorted
|
|
case-insensitively by file name — deterministic queueing order; users
|
|
order albums with `01`-style file name prefixes as everywhere else.
|
|
- Directories are `LibraryNodeChild { is_queable: true }`; queueing one
|
|
resolves its whole subtree via the **default** `resolve_tracks_into`
|
|
walk (one chunk per directory — local disk needs no page streaming).
|
|
Empty directories are fine: they contribute nothing.
|
|
- Fresh read on every navigation, no cache, no file watching — edits with
|
|
a file manager appear on the next visit.
|
|
|
|
### D5 — New crate `fsdy`, non-fatal init, orchestrator routing
|
|
|
|
- New workspace crate **`fsdy`** (naming symmetry with `tidaldy`),
|
|
`PROVIDER_ROOT = "/fs"`, implementing `ProviderClient`.
|
|
- `ProviderOrchestrator` gains `fs_client: Option<Arc<fsdy::Client>>` and
|
|
routes `/fs` prefixes in every trait method; `get_lib_root` adds the
|
|
`/fs` child only when the client exists. **Init failure is non-fatal**
|
|
(warn + run without `/fs`): unlike Tidal, a broken local config must not
|
|
take the whole server down, and existing installations have no
|
|
`fsdy.toml` yet. All I/O through `tokio::fs` (no blocking the runtime).
|
|
|
|
### D6 — Out of scope (explicitly)
|
|
|
|
- `create/rename/delete_lib_node`: `NotSupported`. Track files are edited
|
|
with normal file tools; a TUI editor for them is future work.
|
|
- Reading audio-file tags (ID3 etc.) to synthesize track nodes for plain
|
|
`.mp3` files sitting in the tree: future work — this feature is about
|
|
the serialized-node format.
|
|
- Multiple roots, file watching, link chains: rejected above.
|
|
|
|
## Structure
|
|
|
|
```d2
|
|
direction: right
|
|
|
|
disk: Local disk {
|
|
shape: cylinder
|
|
tree: "root dir: dirs, *.track.toml"
|
|
}
|
|
|
|
server: crabidy-server {
|
|
playback: Playback loop
|
|
orch: ProviderOrchestrator {
|
|
route: "route by first path segment"
|
|
}
|
|
}
|
|
|
|
fsdy: fsdy::Client {
|
|
parse: "parse + validate .track.toml"
|
|
map: "path <-> root-relative file (encoded segments)"
|
|
}
|
|
|
|
tidaldy: tidaldy::Client
|
|
|
|
player: audio-player {
|
|
url: "http(s) -> stream-download"
|
|
file: "other -> File::open"
|
|
}
|
|
|
|
server.playback -> server.orch: "GetTrackUrls(track.path)"
|
|
server.orch -> fsdy: "/fs/..."
|
|
server.orch -> tidaldy: "/tidal/..."
|
|
fsdy -> disk.tree: tokio::fs
|
|
server.playback -> player: "play(url | file path)"
|
|
```
|
|
|
|
## Key flow: queue a directory containing all three playable kinds
|
|
|
|
```d2
|
|
shape: sequence_diagram
|
|
tui: TUI
|
|
pb: Playback loop
|
|
orch: Orchestrator
|
|
fs: fsdy
|
|
tidal: tidaldy
|
|
|
|
tui -> pb: "ReplaceQueue([/fs/mix])"
|
|
pb -> orch: ResolveTracks("/fs/mix", chunk_tx)
|
|
orch -> fs: resolve_tracks_into (spawned)
|
|
fs -> fs: "list dir, parse 3 track files"
|
|
fs -> pb: "chunk of 3 Tracks (paths below)" {style.bold: true}
|
|
pb -> orch: "GetTrackUrls(/fs/mix/a.track.toml)"
|
|
orch -> fs: get_urls_for_track
|
|
fs -> pb: "[/home/u/Music/a.flac]"
|
|
pb -> orch: "GetTrackUrls(/tidal/...) # link track, rewritten path"
|
|
orch -> tidal: get_urls_for_track
|
|
tidal -> pb: "[https://tidal-cdn/...]"
|
|
```
|
|
|
|
(The second track's `Track.path` stays `/fs/...` — its playable is a URL,
|
|
returned by `fsdy::get_urls_for_track`. Only `link` files rewrite the
|
|
path.)
|
|
|
|
## Risks and open questions
|
|
|
|
- **Malicious/odd trees**: deep nesting is bounded only by the walk's
|
|
worklist (memory-cheap); huge directories list in one node — accepted
|
|
for local disk. Traversal and symlink escapes are closed by D4.
|
|
- **Dangling references**: dead `file`/`url`/`link` targets surface at
|
|
play time as the existing "failed to open / no provider owns" warnings;
|
|
the queue keeps going. No preflight validation by design.
|
|
- **Metadata drift** on link tracks (file says X, target now titled Y):
|
|
accepted; the file is the user's curated metadata.
|
|
- Open (future): tag-reading for bare audio files; multiple roots; a
|
|
`%`-style creator that writes a `.track.toml` from inside the TUI.
|