clusterfuck
/
crabidy
4
1
Fork 0
Code Issues 15 Pull Requests Actions Packages Projects 1 Releases 5 Wiki Activity

Add the /orphans provider: a store garbage-collection view 

The content store never shrinks on its own — deleting a save removes only
tomls, never store audio (D7) — so unreferenced audio accumulates. The new
/orphans provider surfaces it for reclamation (realizing store D10).

It lists every store entry, walks the mounted file providers (the /crabidy
tree and /fs) to cross off entries a Playable::Store toml still references,
and presents the rest. Each orphan is an editable + deletable + queueable
child node, so the existing e/d/queue gestures work unchanged — no proto,
TUI, or web change. Rename moves both the audio file and its
.cbd-store.toml sidecar (keeping the derived index in sync); delete removes
both from disk; queueing plays straight from the store.

Enumeration/rename/delete are CrabidyStore methods (it owns the store root
and index); a thin OrphansProvider computes the reference roots and
delegates. fsdy::Client gains a disk_root() accessor so the /fs root can be
handed in as a reference root. Mounts only when the store is present.

Includes the dev-flow artifacts (architecture/quality/plan) and docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Test User 2026-07-22 22:47:23 +02:00
parent 7eaa8fa9b4
commit cd33b790b5
11 changed files with 1287 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -10,8 +10,10 @@ each mounted as a subtree of one library:
├── tidal Tidal streaming (see tidaldy/README.md)
├── youtube YouTube search & playlists (see ytdy/README.md)
├── fs a local music folder (see fsdy/README.md)
└── crabidy your saves: queues, bookmarks (`w`), and captures (`W`),
managed by the server (see architecture/crabidy-store.md)
├── crabidy your saves: queues, bookmarks (`w`), and captures (`W`),
│ managed by the server (see architecture/crabidy-store.md)
└── orphans store audio no save references any more — rename, delete,
or queue it (see architecture/orphans.md)
```
## Binaries

242
architecture/orphans.md Normal file
View File

@ -0,0 +1,242 @@
# The `orphans` provider — a store garbage-collection view
## Context and problem statement
The content-addressed store (`architecture/crabidy-store.md`) never shrinks on
its own. Capturing writes audio + a `<name>.cbd-store.toml` sidecar into
`~/.local/share/crabidy/`; deleting a save (or the `current` queue rolling over,
or a `scan --capture` toml being removed) only ever removes the *toml that
pointed at* a store entry — never the entry itself (store `D7`). That is
deliberate: a store entry may be shared by many tomls, so no single deletion can
know it is safe to reclaim. The consequence, called out as future work in store
`D10`, is that store entries accumulate that **no toml references any more**.
There is today no way to see them or reclaim their disk.
This design adds a read-mostly management provider, **`/orphans`**, that surfaces
exactly those unreferenced store entries and lets the user rename or delete them,
or queue them for a listen before deciding. It is the store's garbage-collection
UI, expressed as an ordinary library subtree so it needs no new client concepts.
This **realizes** store `D10` (orphan reclamation). It changes nothing about how
captures are written or de-duplicated; it only *reads* the store's residue and
offers targeted rename/delete.
## Assumptions
- **Confirmed by the request.** `/orphans` lists every store item, walks all
local file providers, crosses off referenced items, and presents the rest;
entries are renamable (audio file *and* sidecar), deletable (files on disk),
and queueable.
- **"Referenced" means reachable through a mounted local file provider.** The
reference scan walks the disk roots of the running file providers — the
`/crabidy` toml tree (`~/.local/state/crabidy/`, which holds `current`, every
save, every capture) and the `/fs` root (which can hold `Playable::Store`
tomls written by `scan --capture`/`--move`). A `.cbd-track.toml` that lives
*outside* every mounted provider root (e.g. `scan --capture` run on a folder
that is not under `/fs`) is invisible to the walk, so its target counts as an
orphan. This is the only sound definition available without a global
reference index, and it matches the request's wording ("walks all local file
providers"). It is a documented boundary, not a bug (see Risks).
- **Single writer.** As with the rest of the store, one process owns both roots
and serializes mutations under the store's index mutex; there is no concurrent
external writer.
- **Orphan-ness is recomputed on every listing.** There is no persisted orphan
list — consistent with `fsdy`'s "read the tree fresh every visit" philosophy.
A capture that adds a reference makes an entry stop being an orphan on the
next listing.
## What is a "store item" and when is it an orphan
- A **store item** is a pair in `store_root`: an audio file `<name>` and its
sidecar `<name>.cbd-store.toml`. The sidecar is the source of truth (store
`D2`); the set of items is the set of sidecars that have a readable audio file
beside them. (A sidecar without audio, or audio without a sidecar, is
malformed residue — reported so it can be reclaimed too; see D4.)
- A store item `<name>` is **referenced** iff some `.cbd-track.toml` under a
mounted file-provider root validates to `Playable::Store(<name>)`.
- **Orphans = all store items − referenced store items.**
Because `/orphans` only ever exposes unreferenced entries, renaming or deleting
one **cannot break any toml reference** — that is what makes the destructive
operations safe by construction (subject to the narrow race in Risks).
## Options considered
### Presentation: tracks vs. child nodes
Each orphan must be renamable, deletable, and queueable. The wire has two
carriers with capability flags:
- **`Track`** — has `is_captured`/`is_skipped` but **no** `is_editable`. Tracks
can be deleted (`tracks_deletable`) and queued, but the library has **no
rename-a-track gesture** anywhere; adding one means new proto surface plus TUI
and web changes.
- **`LibraryNodeChild`** — already carries `is_editable`, `is_deletable`,
`is_queable`, `is_captured`. The node-editing feature already binds `e` →
`rename_lib_node(child_path, new_title)` and `d` → `delete_lib_node(child_path)`
for children that advertise the flags (that is how `/tidal/search` terms and
`/crabidy` saves are renamed/deleted today), and queueing a queueable child
resolves its tracks.
**Decision: present each orphan as an editable + deletable + queueable child
node** of `/orphans`, titled by its store file `<name>` (the thing a rename
edits). Entering the node lists its single track (the store audio, with metadata
from the sidecar); queueing the node — or the whole `/orphans` root — resolves
that track. This reuses `rename_lib_node`/`delete_lib_node`/`resolve_tracks_into`
and every client gesture **with zero proto, TUI, or web changes**. The only
minor wart — each orphan is a one-track "folder" — is acceptable and is exactly
how a single-track save already presents. The track-carrier option was rejected
purely on cost: it buys nothing the node model lacks and forces a wire change
just to gain a rename gesture.
### Home of the logic: new crate vs. server module
The orphan computation needs the store root and index (to enumerate items and
mutate them) *and* the file-provider disk roots (to find references). A standalone
`orphandy` crate would have to duplicate `CrabidyStore` internals it does not own.
**Decision: keep it in `crabidy-server`.** Orphan enumeration, rename, and delete
become methods on `CrabidyStore` (it already owns `store_root` and the index). A
thin new `OrphansProvider` (`crabidy-server/src/orphans.rs`) implements
`ProviderClient`, holding `Arc<CrabidyStore>` plus the list of reference roots to
walk, and delegates to those store methods. `ProviderOrchestrator` mounts and
routes `/orphans` exactly like the other providers.
## Boundaries and interfaces
```d2
direction: right
tui: TUI / web / cbd-cli { shape: person }
orchestrator: ProviderOrchestrator {
routes: "routes /orphans/*"
}
orphans: OrphansProvider {
refroots: "ref_roots: Vec<PathBuf>"
}
store: CrabidyStore {
index: "StoreIndex (by_hash / by_provider_id)"
ops: "list_orphans / rename_orphan / delete_orphan"
}
data: content store\n~/.local/share/crabidy {
shape: cylinder
items: "<name> + <name>.cbd-store.toml"
}
crabidytree: /crabidy tree\n~/.local/state/crabidy { shape: cylinder }
fstree: /fs root { shape: cylinder }
tui -> orchestrator: "get / rename / delete / queue /orphans/*"
orchestrator -> orphans: delegate
orphans -> store: "enumerate + mutate (by name)"
store -> data: "read sidecars, rename/delete files"
orphans -> crabidytree: "walk for Playable::Store refs"
orphans -> fstree: "walk for Playable::Store refs"
```
### The orphan diff (what a listing computes)
```d2
direction: down
allitems: "all store items\n(scan *.cbd-store.toml in store_root)"
refs: "referenced set\n(walk ref_roots for\nPlayable::Store(name))"
diff: "orphans = all − referenced" { shape: diamond }
node: "/orphans node:\none editable/deletable/queueable\nchild per orphan"
allitems -> diff
refs -> diff
diff -> node
```
### Provider surface (`OrphansProvider: ProviderClient`)
Mounted at `/orphans` only when `crabidy_store` is present (it is the store's
view). Paths: the root `/orphans`, and one child per orphan at
`/orphans/<encode_segment(name)>`. There are no deeper levels.
- `get_lib_root` / `get_lib_node("/orphans")` — a queueable, non-creatable node
whose children are the current orphans (recomputed by the diff above). Each
child: `title = <name>`, `is_queable = true`, `is_editable = true`,
`is_deletable = true`, `is_downloadable = false`, `is_captured = true`.
- `get_lib_node("/orphans/<seg>")` — a queueable, childless node carrying the
single `Track` for that store entry (metadata from the sidecar's first
provider entry; `is_captured = true`). Unknown/renamed-away segment →
`MalformedPath`.
- `is_track_path` — always `false`: orphans are addressed as nodes, and the one
track is reached by resolving the node (so the default `resolve_tracks_into`
walk queues it). `get_metadata_for_track` therefore is not the entry point;
`get_urls_for_track("/orphans/<seg>")` returns `store_root/<name>` (a local
file path, exactly like a resolved `Playable::Store`) so the resolved track
still plays.
- `rename_lib_node("/orphans/<seg>", new)` — validates `new` as a bare store
file name (`validate_folder_name(new, &[])`: non-empty, no separators/NUL, no
leading dot), refuses a name already taken by another store entry
(`InvalidInput`), then renames **both** `<old>`→`<new>` audio and
`<old>.cbd-store.toml`→`<new>.cbd-store.toml`, and updates the in-memory index
(drop the old name's mappings, re-insert under the new name; hash and provider
ids are unchanged). Returns the renamed node at `/orphans/<encode(new)>`.
- `delete_lib_node("/orphans/<seg>")` — removes the audio file and the sidecar
from `store_root` and drops the entry from the index; returns the refreshed
`/orphans` root. Idempotent (an already-gone entry succeeds).
- `create_lib_node` — `NotSupported` (the root is not creatable).
### Store methods added to `CrabidyStore`
- `list_orphans(&self, ref_roots: &[PathBuf]) -> Result<Vec<OrphanEntry>, StoreError>`
— scan `store_root` for `*.cbd-store.toml`; build the referenced set by walking
each `ref_root` recursively for `*.cbd-track.toml` and collecting
`Playable::Store(name)`; return the difference as `OrphanEntry { name, title,
artist, duration, album }` (metadata from the sidecar's first provider entry).
- `orphan_track(&self, name) -> Result<Track, StoreError>` /
`orphan_url(&self, name) -> Result<String, StoreError>` — build the wire track
/ resolve the store audio path for a single entry.
- `rename_orphan(&self, old, new)` / `delete_orphan(&self, name)` — the mutations
above, under the index mutex, with the index kept in sync.
- `StoreIndex::remove(&mut self, name, sidecar)` — the inverse of `insert`, so
rename/delete can update the derived index without a full rescan.
### Reference roots wiring
`OrphansProvider` is constructed in `ProviderOrchestrator::init` with
`ref_roots` = the disk roots of the mounted file providers: the `/crabidy` tree
(`store.tree_dir()`) and, when enabled, the `/fs` root. A new
`fsdy::Client::disk_root(&self) -> &Path` accessor exposes the `/fs` root (the
`/crabidy` tree root is already available via `CrabidyStore::tree_dir`). If more
`fsdy` instances are ever mounted, they are added to this list — the definition
of "local file provider" is "an `fsdy` instance whose root can hold store
references."
## Risks and open questions
- **Capture-then-delete race (TOCTOU).** Between a `/orphans` listing and a
delete, a concurrent `W` capture could hash-hit the very entry the user is
about to delete and write a fresh `Playable::Store` reference to it; deleting
then leaves that new toml dangling. The window is small (store mutations
serialize under the index mutex and orphan-ness is recomputed every listing),
and the failure is benign: a dangling store reference already resolves to
`MalformedPath` at play time and is skipped, not a crash. Accepted; noted here
rather than engineered away.
- **References outside mounted roots are not counted.** As stated in Assumptions,
a `scan --capture` toml under a folder that is not mounted under `/fs` will not
be seen, so its target shows as an orphan. Deleting it would orphan that
toml's audio. The mitigation is scope discipline (scan under `/fs`); the
alternative — a persisted global reference index — is out of scope and would
fight the "read fresh" design.
- **Cost.** A `/orphans` listing scans the whole store plus both trees on every
visit (no cache), i.e. O(store entries + tomls under the roots). This matches
`fsdy`'s existing per-visit read cost and is fine for personal-library sizes;
if it ever bites, memoizing behind the index's mutation counter is the escape
hatch. Not premature-optimized here.
- **Malformed residue.** A sidecar with no audio (or vice versa) is itself
reclaimable junk. `list_orphans` reports such half-entries as orphans (titled
by whatever is present) so a delete cleans them up; it never treats a
half-entry as "referenced."
- **Open question:** should the `/orphans` root also expose a single "delete all"
affordance? Deferred — per-item delete covers the request; bulk reclaim can be
a later addition (a client could multi-select and delete, once marks exist
there).

332
crabidy-server/src/crabidy_store.rs
View File

@ -75,6 +75,16 @@ pub enum StoreError {
TomlWrite(#[from] toml::ser::Error),
#[error("http client: {0}")]
Http(#[from] reqwest::Error),
/// No store entry with the given name (rename/delete of a vanished
/// orphan).
#[error("no such store entry: {0}")]
NotFound(String),
/// A rename target name is already used by another store entry.
#[error("store name already in use: {0}")]
NameTaken(String),
/// A rename target name is not a legal bare store file name.
#[error("invalid store name: {0}")]
InvalidName(String),
}
/// One `(provider, id)` identity that resolves to a store entry, plus the
@ -200,6 +210,39 @@ impl StoreIndex {
}
}
}
/// The inverse of [`Self::insert`]: drops every mapping that still points
/// at `name` for this sidecar's hash and provider identities. A no-op for
/// mappings already reassigned to another entry. Used by orphan
/// rename/delete to keep the derived index in sync without a full rescan.
pub fn remove(&mut self, name: &StoreName, sidecar: &StoreSidecar) {
if self.by_hash.get(&sidecar.hash).is_some_and(|n| n == name) {
self.by_hash.remove(&sidecar.hash);
}
for provider in &sidecar.providers {
if provider.id.is_empty() {
continue;
}
let key = (provider.provider.clone(), provider.id.clone());
if self.by_provider_id.get(&key).is_some_and(|n| n == name) {
self.by_provider_id.remove(&key);
}
}
}
}
/// One unreferenced store entry surfaced by the `/orphans` provider (see
/// `architecture/orphans.md`). `name` is the bare store file name (the audio
/// file, and the stem of its `*.cbd-store.toml` sidecar); the rest is display
/// metadata from the sidecar's first provider entry (or the name itself for a
/// sidecar-less audio file).
#[derive(Debug, Clone)]
pub struct OrphanEntry {
pub name: StoreName,
pub title: String,
pub artist: String,
pub duration: Option<u32>,
pub album: Option<fsdy::AlbumMeta>,
}
/// Owns both store roots and serializes all mutation.
@ -936,6 +979,233 @@ impl CrabidyStore {
}
node.is_captured = all_captured && node.children.is_empty();
}
/// Lists store entries that no `.cbd-track.toml` under any `ref_roots`
/// references — the `/orphans` view (architecture/orphans.md). Scans the
/// store for entries (a `*.cbd-store.toml` sidecar and/or a bare audio
/// file), builds the referenced set by walking `ref_roots` for
/// `Playable::Store` links, and returns the difference. Malformed residue
/// (a sidecar without audio, or audio without a sidecar) is reported so it
/// can be reclaimed. Reads fresh every call — there is no cached list.
pub async fn list_orphans(
&self,
ref_roots: &[PathBuf],
) -> Result<Vec<OrphanEntry>, StoreError> {
// Every store item, keyed by bare name → its parsed sidecar (if any).
let mut items: std::collections::HashMap<StoreName, Option<StoreSidecar>> =
std::collections::HashMap::new();
let mut rd = match tokio::fs::read_dir(&self.store_root).await {
Ok(rd) => rd,
Err(err) if err.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()),
Err(err) => return Err(err.into()),
};
while let Some(entry) = rd.next_entry().await? {
let file_name = entry.file_name();
let Some(file_name) = file_name.to_str() else {
continue;
};
// Hidden entries and in-flight download temp files are not items.
if file_name.starts_with('.') {
continue;
}
if let Some(stem) = file_name.strip_suffix(SIDECAR_SUFFIX) {
let sidecar = match tokio::fs::read_to_string(entry.path()).await {
Ok(text) => toml::from_str::<StoreSidecar>(&text).ok(),
Err(err) => {
warn!(sidecar = file_name, "cannot read store sidecar: {err}");
None
}
};
items.entry(stem.to_string()).or_insert(sidecar);
} else {
// A bare audio file; keep any sidecar already recorded for it.
items.entry(file_name.to_string()).or_insert(None);
}
}
let referenced = Self::collect_referenced(ref_roots).await;
let mut orphans: Vec<OrphanEntry> = items
.into_iter()
.filter(|(name, _)| !referenced.contains(name))
.map(|(name, sidecar)| orphan_entry(name, sidecar.as_ref()))
.collect();
orphans.sort_by_key(|orphan| orphan.name.to_lowercase());
Ok(orphans)
}
/// Walks each root (recursively, breadth-first) for `*.cbd-track.toml`
/// files and collects the store names they link to via
/// [`fsdy::Playable::Store`]. Unreadable directories and unparseable track
/// files are skipped with a warning — one bad file never aborts the scan.
async fn collect_referenced(roots: &[PathBuf]) -> std::collections::HashSet<StoreName> {
let mut referenced = std::collections::HashSet::new();
let mut stack: Vec<PathBuf> = roots.to_vec();
while let Some(dir) = stack.pop() {
let mut rd = match tokio::fs::read_dir(&dir).await {
Ok(rd) => rd,
Err(err) if err.kind() == std::io::ErrorKind::NotFound => continue,
Err(err) => {
warn!(dir = %dir.display(), "cannot scan for store references: {err}");
continue;
}
};
while let Ok(Some(entry)) = rd.next_entry().await {
let Ok(file_type) = entry.file_type().await else {
continue;
};
// Symlinks are skipped by design (no cycles, no escaping).
if file_type.is_symlink() {
continue;
}
let path = entry.path();
if file_type.is_dir() {
stack.push(path);
continue;
}
let name = entry.file_name();
let Some(name) = name.to_str() else {
continue;
};
if !name.ends_with(fsdy::TRACK_FILE_SUFFIX) {
continue;
}
let Ok(text) = tokio::fs::read_to_string(&path).await else {
continue;
};
match fsdy::TrackFile::parse(&text).map(|f| f.playable()) {
Ok(Ok(fsdy::Playable::Store(store_name))) => {
referenced.insert(store_name);
}
Ok(_) => {}
Err(err) => {
warn!(file = %path.display(), "skipping invalid track file: {err}")
}
}
}
}
referenced
}
/// The wire [`Track`] for a single store entry `name` at library path
/// `lib_path` — metadata from the sidecar's first provider entry (or the
/// bare name when there is no sidecar). Always `is_captured`, never
/// skipped. [`StoreError::NotFound`] when neither the sidecar nor an audio
/// file exists.
pub async fn orphan_track(&self, name: &str, lib_path: &str) -> Result<Track, StoreError> {
let sidecar = self.read_sidecar(name).await.ok();
if sidecar.is_none() && !self.audio_exists(name).await {
return Err(StoreError::NotFound(name.to_string()));
}
let first = sidecar.as_ref().and_then(|s| s.providers.first());
Ok(Track {
path: lib_path.to_string(),
artist: first.map(|p| p.artist.clone()).unwrap_or_default(),
title: first
.map(|p| p.title.clone())
.unwrap_or_else(|| name.to_string()),
duration: first.and_then(|p| p.duration),
album: first.and_then(|p| p.album.as_ref()).map(|a| Album {
title: a.title.clone(),
release_date: a.release_date.clone(),
}),
is_skipped: false,
provider_item_id: first.map(|p| p.id.clone()).unwrap_or_default(),
is_captured: true,
})
}
/// The local audio path a single store entry resolves to (the file the
/// player opens), or [`StoreError::NotFound`]/[`StoreError::InvalidName`].
pub async fn orphan_url(&self, name: &str) -> Result<String, StoreError> {
if name.is_empty() || name.contains(['/', '\\', '\0']) {
return Err(StoreError::InvalidName(name.to_string()));
}
if !self.audio_exists(name).await {
return Err(StoreError::NotFound(name.to_string()));
}
self.store_root
.join(name)
.to_str()
.map(str::to_string)
.ok_or_else(|| StoreError::InvalidName(name.to_string()))
}
/// Renames a store entry: the audio `<old>`→`<new>` and its
/// `<old>.cbd-store.toml` sidecar → `<new>.cbd-store.toml`, keeping the
/// derived index in sync. `new` must be a legal bare store name and must
/// not already be taken. Serialized under the index mutex; safe because
/// `/orphans` only exposes unreferenced entries (architecture/orphans.md).
pub async fn rename_orphan(&self, old: &str, new: &str) -> Result<(), StoreError> {
let new = fsdy::validate_folder_name(new, &[])
.map_err(|_| StoreError::InvalidName(new.to_string()))?;
let mut index = self.index.lock().await;
if !self.audio_exists(old).await && !self.sidecar_exists(old).await {
return Err(StoreError::NotFound(old.to_string()));
}
if new != old && self.name_taken(new).await {
return Err(StoreError::NameTaken(new.to_string()));
}
if new == old {
return Ok(());
}
self.rename_path(&self.store_root.join(old), &self.store_root.join(new))
.await?;
self.rename_path(&self.sidecar_path(old), &self.sidecar_path(new))
.await?;
// Re-point the index: read the moved sidecar (if any) and swap names.
if let Ok(sidecar) = self.read_sidecar(new).await {
index.remove(&old.to_string(), &sidecar);
index.insert(&new.to_string(), &sidecar);
}
Ok(())
}
/// Deletes a store entry's audio file and sidecar and drops it from the
/// index. Idempotent: an already-gone entry succeeds. Serialized under the
/// index mutex.
pub async fn delete_orphan(&self, name: &str) -> Result<(), StoreError> {
if name.is_empty() || name.contains(['/', '\\', '\0']) {
return Err(StoreError::InvalidName(name.to_string()));
}
let mut index = self.index.lock().await;
let sidecar = self.read_sidecar(name).await.ok();
remove_file_if_present(&self.store_root.join(name)).await?;
remove_file_if_present(&self.sidecar_path(name)).await?;
if let Some(sidecar) = sidecar {
index.remove(&name.to_string(), &sidecar);
}
Ok(())
}
/// Whether a bare-named audio file exists in the store root.
async fn audio_exists(&self, name: &str) -> bool {
tokio::fs::try_exists(self.store_root.join(name))
.await
.unwrap_or(false)
}
/// Whether a store entry's sidecar exists.
async fn sidecar_exists(&self, name: &str) -> bool {
tokio::fs::try_exists(self.sidecar_path(name))
.await
.unwrap_or(false)
}
/// Renames one file, falling back to copy+remove across devices; a missing
/// source is a no-op (the counterpart may be a sidecar-less entry).
async fn rename_path(&self, from: &Path, to: &Path) -> Result<(), StoreError> {
if !tokio::fs::try_exists(from).await.unwrap_or(false) {
return Ok(());
}
match tokio::fs::rename(from, to).await {
Ok(()) => Ok(()),
Err(_) => {
tokio::fs::copy(from, to).await?;
let _ = tokio::fs::remove_file(from).await;
Ok(())
}
}
}
}
/// Whether the audio bytes came from a local file (hashed in place) or a
@ -979,6 +1249,30 @@ fn provider_entry(track: &Track, provider: &str, id: &str) -> ProviderEntry {
}
}
/// Builds an [`OrphanEntry`] from a store name and its sidecar (if any),
/// taking display metadata from the sidecar's first provider entry.
fn orphan_entry(name: StoreName, sidecar: Option<&StoreSidecar>) -> OrphanEntry {
let first = sidecar.and_then(|s| s.providers.first());
OrphanEntry {
title: first
.map(|p| p.title.clone())
.unwrap_or_else(|| name.clone()),
artist: first.map(|p| p.artist.clone()).unwrap_or_default(),
duration: first.and_then(|p| p.duration),
album: first.and_then(|p| p.album.clone()),
name,
}
}
/// Removes a file, treating an already-absent file as success.
async fn remove_file_if_present(path: &Path) -> Result<(), StoreError> {
match tokio::fs::remove_file(path).await {
Ok(()) => Ok(()),
Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()),
Err(err) => Err(err.into()),
}
}
/// Sanitizes a title into a bare file stem: path separators and NUL become
/// `_`, leading dots stripped, trimmed, falling back to `track`.
fn sanitize_stem(title: &str) -> String {
@ -1511,6 +1805,44 @@ mod tests {
assert_eq!(entries(store.store_dir()).len(), 2, "no second entry");
}
#[tokio::test]
async fn rename_orphan_moves_both_files_and_repoints_the_index() {
let (store, dir) = open_store().await;
let src = write_bytes(dir.path(), "src.flac", b"AAAA").await;
let mut mock = MockProvider::default();
let t = track("/tidal/album/1/100", "100", "Song");
mock.tracks.insert(t.path.clone(), (t.clone(), src));
mock.nodes.insert(
"/tidal/album/1".to_string(),
node("/tidal/album/1", vec![t]),
);
let progress = Progress::silent("x", true);
store
.save(&mock, "/tidal/album/1", "s1", SaveMode::Capture, &progress)
.await
.expect("capture");
let old = store_name_in_save(&store, "s1").await;
store
.rename_orphan(&old, "renamed.flac")
.await
.expect("rename");
// Both files moved to the new name.
assert!(store.store_dir().join("renamed.flac").is_file());
assert!(store
.store_dir()
.join(format!("renamed.flac{SIDECAR_SUFFIX}"))
.is_file());
assert!(!store.store_dir().join(&old).exists());
// The index now resolves the provider id to the new name (so a later
// capture de-dups against the renamed entry).
let index = store.index.lock().await;
assert_eq!(
index.by_provider_id("tidal", "100"),
Some(&"renamed.flac".to_string())
);
}
#[tokio::test]
async fn ingest_file_move_removes_the_source() {
let (store, dir) = open_store().await;

1
crabidy-server/src/lib.rs
View File

@ -5,6 +5,7 @@ pub mod web;
pub mod capture;
pub mod cli;
pub mod crabidy_store;
pub mod orphans;
pub mod playback;
pub mod provider;
pub mod rpc;

378
crabidy-server/src/orphans.rs Normal file
View File

@ -0,0 +1,378 @@
//! The `/orphans` provider: a store garbage-collection view.
//!
//! `/orphans` surfaces content-store entries that no mounted local file
//! provider references any more (see `architecture/orphans.md`). It is a
//! read-mostly management subtree: each orphan is presented as an editable,
//! deletable, queueable child node, so every client's existing `e`/`d`/queue
//! gestures work unchanged (no proto surface is added).
//!
//! The heavy lifting lives on [`CrabidyStore`] (it owns the store root and the
//! derived index); this provider only supplies the reference roots to walk and
//! translates between library paths and store names.
use std::path::PathBuf;
use std::sync::Arc;
use async_trait::async_trait;
use crabidy_core::{
proto::crabidy::{LibraryNode, LibraryNodeChild, Track},
ProviderClient, ProviderError,
};
use tracing::warn;
use crate::crabidy_store::{CrabidyStore, StoreError};
/// The single library segment this provider owns.
pub const ORPHANS_PROVIDER_ROOT: &str = "/orphans";
/// Maps a [`StoreError`] to the provider-boundary error: a missing entry or a
/// malformed name is a bad path, a taken/invalid rename target is bad input,
/// everything else is internal.
fn to_provider_error(err: StoreError) -> ProviderError {
match err {
StoreError::NotFound(_) => ProviderError::MalformedPath,
StoreError::NameTaken(_) | StoreError::InvalidName(_) => ProviderError::InvalidInput,
other => {
warn!("orphans store error: {other}");
ProviderError::InternalError
}
}
}
/// The `/orphans` provider.
///
/// Holds the store (for enumeration and mutation) and the disk roots of every
/// mounted local file provider (for the reference scan). Constructed in
/// [`crate::provider::ProviderOrchestrator::init`] only when the store is
/// present.
#[derive(Debug)]
pub struct OrphansProvider {
/// The content store: enumerates orphans and performs rename/delete.
store: Arc<CrabidyStore>,
/// Disk roots to walk for `Playable::Store` references — the `/crabidy`
/// toml tree and (when enabled) the `/fs` root. "Referenced" is defined as
/// reachable through one of these (architecture/orphans.md).
ref_roots: Vec<PathBuf>,
}
impl OrphansProvider {
/// Builds the provider over `store`, scanning `ref_roots` for references.
pub fn new(store: Arc<CrabidyStore>, ref_roots: Vec<PathBuf>) -> Self {
Self { store, ref_roots }
}
/// The bare store name addressed by an `/orphans/<seg>` path (its single
/// decoded segment); `None` for the bare root, a deeper path, or a segment
/// that is not a legal bare name (empty, `.`, `..`, or containing a
/// separator) — which keeps a crafted path from escaping the store root.
fn entry_name(&self, path: &str) -> Option<String> {
let rest = path
.strip_prefix(ORPHANS_PROVIDER_ROOT)?
.strip_prefix('/')?;
if rest.is_empty() || rest.contains('/') {
return None;
}
let name = crabidy_core::decode_segment(rest);
if name.is_empty() || name == "." || name == ".." || name.contains(['/', '\\', '\0']) {
return None;
}
Some(name)
}
/// The library path of the orphan named `name`.
fn entry_path(name: &str) -> String {
crabidy_core::join_path(ORPHANS_PROVIDER_ROOT, &crabidy_core::encode_segment(name))
}
/// The `/orphans` root node, its children recomputed from the current
/// orphan set.
async fn root_node(&self) -> Result<LibraryNode, ProviderError> {
let orphans = self
.store
.list_orphans(&self.ref_roots)
.await
.map_err(to_provider_error)?;
let children = orphans
.into_iter()
.map(|orphan| {
let mut child =
LibraryNodeChild::new(Self::entry_path(&orphan.name), orphan.name, true);
// Reuse the existing rename/delete/queue gestures: an orphan is
// an editable, deletable, queueable, already-captured child.
child.is_editable = true;
child.is_deletable = true;
child.is_downloadable = false;
child.is_captured = true;
child
})
.collect();
Ok(LibraryNode {
path: ORPHANS_PROVIDER_ROOT.to_string(),
title: "orphans".to_string(),
children,
parent: Some(crabidy_core::ROOT_PATH.to_string()),
tracks: Vec::new(),
is_queable: true,
is_creatable: false,
is_downloadable: false,
tracks_deletable: false,
is_captured: false,
})
}
/// A single orphan's node: childless, queueable, carrying its one track.
async fn entry_node(&self, path: &str, name: &str) -> Result<LibraryNode, ProviderError> {
let track = self
.store
.orphan_track(name, path)
.await
.map_err(to_provider_error)?;
Ok(LibraryNode {
path: path.to_string(),
title: name.to_string(),
children: Vec::new(),
parent: Some(ORPHANS_PROVIDER_ROOT.to_string()),
tracks: vec![track],
is_queable: true,
is_creatable: false,
is_downloadable: false,
tracks_deletable: false,
is_captured: true,
})
}
}
#[async_trait]
impl ProviderClient for OrphansProvider {
/// Not constructed from a config string — built by the orchestrator with a
/// store handle. Present only to satisfy the trait; never called.
async fn init(_raw_toml_settings: &str) -> Result<Self, ProviderError> {
Err(ProviderError::NotSupported)
}
fn settings(&self) -> String {
String::new()
}
/// Always false: orphans are addressed as nodes. The one track per orphan
/// is reached by resolving the node (the default `resolve_tracks_into`
/// walk), not by a track path.
fn is_track_path(&self, _path: &str) -> bool {
false
}
/// Resolves an orphan's audio to its store file path (a local file, exactly
/// like a resolved `Playable::Store`), so the walked-out track still plays.
async fn get_urls_for_track(&self, track_path: &str) -> Result<Vec<String>, ProviderError> {
let name = self
.entry_name(track_path)
.ok_or(ProviderError::MalformedPath)?;
let url = self
.store
.orphan_url(&name)
.await
.map_err(to_provider_error)?;
Ok(vec![url])
}
/// Track metadata for a single orphan (from the sidecar's first provider
/// entry); `is_captured` is true.
async fn get_metadata_for_track(&self, track_path: &str) -> Result<Track, ProviderError> {
let name = self
.entry_name(track_path)
.ok_or(ProviderError::MalformedPath)?;
self.store
.orphan_track(&name, track_path)
.await
.map_err(to_provider_error)
}
/// A minimal `/orphans` root; children are discovered via
/// [`Self::get_lib_node`] (listing needs async store access).
fn get_lib_root(&self) -> LibraryNode {
LibraryNode {
path: ORPHANS_PROVIDER_ROOT.to_string(),
title: "orphans".to_string(),
children: Vec::new(),
parent: Some(crabidy_core::ROOT_PATH.to_string()),
tracks: Vec::new(),
is_queable: true,
is_creatable: false,
is_downloadable: false,
tracks_deletable: false,
is_captured: false,
}
}
/// The `/orphans` root lists one child per current orphan; an
/// `/orphans/<seg>` path returns that orphan's single-track node.
/// Recomputes the orphan set on every call (no cache).
async fn get_lib_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
if path == ORPHANS_PROVIDER_ROOT {
return self.root_node().await;
}
let name = self.entry_name(path).ok_or(ProviderError::MalformedPath)?;
self.entry_node(path, &name).await
}
/// The root is not creatable.
async fn create_lib_node(
&self,
_parent_path: &str,
_title: &str,
) -> Result<LibraryNode, ProviderError> {
Err(ProviderError::NotSupported)
}
/// Renames an orphan's store files (audio + sidecar), keeping the index in
/// sync; returns the node at its new `/orphans/<encode(new)>` path.
async fn rename_lib_node(
&self,
path: &str,
new_title: &str,
) -> Result<LibraryNode, ProviderError> {
let old = self.entry_name(path).ok_or(ProviderError::NotSupported)?;
self.store
.rename_orphan(&old, new_title)
.await
.map_err(to_provider_error)?;
let new_path = Self::entry_path(new_title.trim());
let name = self
.entry_name(&new_path)
.ok_or(ProviderError::InvalidInput)?;
self.entry_node(&new_path, &name).await
}
/// Deletes an orphan's audio file and sidecar; returns the refreshed
/// `/orphans` root. Idempotent.
async fn delete_lib_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
let name = self.entry_name(path).ok_or(ProviderError::NotSupported)?;
self.store
.delete_orphan(&name)
.await
.map_err(to_provider_error)?;
self.root_node().await
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::crabidy_store::{ProviderEntry, StoreSidecar};
use std::path::Path;
use tempfile::TempDir;
async fn open_store() -> (Arc<CrabidyStore>, TempDir) {
let dir = TempDir::new().expect("tempdir");
let store = CrabidyStore::open(dir.path().join("state"), dir.path().join("store"))
.await
.expect("open store");
(Arc::new(store), dir)
}
/// Writes a store entry (audio + sidecar) directly, bypassing capture.
async fn write_entry(store: &CrabidyStore, name: &str, hash: &str, id: &str, title: &str) {
let root = store.store_dir();
tokio::fs::write(root.join(name), b"AUDIO")
.await
.expect("audio");
let sidecar = StoreSidecar {
hash: hash.to_string(),
providers: vec![ProviderEntry {
provider: "tidal".to_string(),
id: id.to_string(),
title: title.to_string(),
artist: "artist".to_string(),
duration: Some(123),
album: None,
aliases: Vec::new(),
}],
};
let text = toml::to_string_pretty(&sidecar).expect("ser");
tokio::fs::write(root.join(format!("{name}.cbd-store.toml")), text)
.await
.expect("sidecar");
}
/// Writes a `.cbd-track.toml` under `dir` that references store `name`.
async fn write_store_reference(dir: &Path, file: &str, name: &str) {
tokio::fs::create_dir_all(dir).await.expect("mkdir");
let toml = format!("title = \"ref\"\n\n[playable]\nstore = \"{name}\"\n");
tokio::fs::write(dir.join(file), toml)
.await
.expect("ref toml");
}
#[tokio::test]
async fn orphans_are_store_entries_no_reference_reaches() {
let (store, dir) = open_store().await;
write_entry(&store, "a.flac", "blake3:a", "1", "A").await;
write_entry(&store, "b.flac", "blake3:b", "2", "B").await;
// A reference under a scanned root rescues b.flac only.
let ref_root = dir.path().join("tree");
write_store_reference(&ref_root, "0001 b.cbd-track.toml", "b.flac").await;
let provider = OrphansProvider::new(store, vec![ref_root]);
let root = provider.get_lib_node("/orphans").await.expect("root");
let names: Vec<&str> = root.children.iter().map(|c| c.title.as_str()).collect();
assert_eq!(names, vec!["a.flac"], "only the unreferenced entry orphans");
let child = &root.children[0];
assert!(child.is_editable && child.is_deletable && child.is_queable && child.is_captured);
assert!(!child.is_downloadable);
}
#[tokio::test]
async fn a_leaf_carries_one_captured_track_that_resolves_to_store_audio() {
let (store, _dir) = open_store().await;
write_entry(&store, "song.flac", "blake3:s", "9", "Song").await;
let provider = OrphansProvider::new(store.clone(), vec![]);
let path = OrphansProvider::entry_path("song.flac");
let node = provider.get_lib_node(&path).await.expect("leaf");
assert!(node.children.is_empty());
assert_eq!(node.tracks.len(), 1);
assert!(node.tracks[0].is_captured);
assert_eq!(node.tracks[0].title, "Song");
let urls = provider.get_urls_for_track(&path).await.expect("urls");
let expected = store.store_dir().join("song.flac");
assert_eq!(urls, vec![expected.to_str().unwrap().to_string()]);
}
#[tokio::test]
async fn rename_moves_both_files_and_delete_removes_them() {
let (store, _dir) = open_store().await;
write_entry(&store, "old.flac", "blake3:x", "7", "Old").await;
let provider = OrphansProvider::new(store.clone(), vec![]);
let path = OrphansProvider::entry_path("old.flac");
let renamed = provider
.rename_lib_node(&path, "new.flac")
.await
.expect("rename");
assert_eq!(renamed.title, "new.flac");
assert!(store.store_dir().join("new.flac").is_file());
assert!(!store.store_dir().join("old.flac").exists());
assert!(store.store_dir().join("new.flac.cbd-store.toml").is_file());
assert!(!store.store_dir().join("old.flac.cbd-store.toml").exists());
let new_path = OrphansProvider::entry_path("new.flac");
provider.delete_lib_node(&new_path).await.expect("delete");
assert!(!store.store_dir().join("new.flac").exists());
assert!(!store.store_dir().join("new.flac.cbd-store.toml").exists());
}
#[tokio::test]
async fn unknown_and_traversal_paths_are_malformed() {
let (store, _dir) = open_store().await;
let provider = OrphansProvider::new(store, vec![]);
// A never-created entry.
assert!(matches!(
provider.get_lib_node("/orphans/ghost.flac").await,
Err(ProviderError::MalformedPath)
));
// A traversal attempt decodes to a rejected segment.
assert!(provider.entry_name("/orphans/..%2Fetc").is_none());
assert!(provider.entry_name("/orphans/a/b").is_none());
assert!(provider.entry_name("/orphans").is_none());
}
}

78
crabidy-server/src/provider.rs
View File

@ -1,4 +1,5 @@
use crate::crabidy_store::{CrabidyStore, SaveMode, CRABIDY_PROVIDER_ROOT, CURRENT_NAME};
use crate::orphans::{OrphansProvider, ORPHANS_PROVIDER_ROOT};
use crate::{ProviderCommand, ProviderMessage};
use async_trait::async_trait;
use crabidy_core::{
@ -25,6 +26,10 @@ pub struct ProviderOrchestrator {
/// tree. `None` disables saving/capturing (and the `/crabidy` mount goes
/// with it).
crabidy_store: Option<Arc<CrabidyStore>>,
/// The `/orphans` provider: the store's garbage-collection view
/// (architecture/orphans.md). `None` whenever `crabidy_store` is — it is a
/// view over the store.
orphans_client: Option<Arc<OrphansProvider>>,
/// The YouTube provider (yt-dlp backed); `None` when the binary
/// probe failed at init (architecture/youtube-provider.md D2).
youtube_client: Option<Arc<ytdy::Client>>,
@ -46,6 +51,11 @@ fn youtube_owns(path: &str) -> bool {
path == ytdy::PROVIDER_ROOT || path.starts_with("/youtube/")
}
/// Whether a path belongs to the `/orphans` provider.
fn orphans_owns(path: &str) -> bool {
path == ORPHANS_PROVIDER_ROOT || path.starts_with("/orphans/")
}
impl ProviderOrchestrator {
/// The fs client, or `MalformedPath` (with a warning) when the
/// provider is disabled — a `/fs` path then has no owner.
@ -70,6 +80,15 @@ impl ProviderOrchestrator {
self.crabidy_store.clone()
}
/// The `/orphans` client, or `MalformedPath` (with a warning) when the
/// store — and so the view — is disabled.
fn orphans_provider(&self) -> Result<&OrphansProvider, ProviderError> {
self.orphans_client.as_deref().ok_or_else(|| {
warn!("orphans provider is disabled");
ProviderError::MalformedPath
})
}
/// The YouTube client, or `MalformedPath` (with a warning) when the
/// provider is disabled — a `/youtube` path then has no owner.
fn youtube_provider(&self) -> Result<&ytdy::Client, ProviderError> {
@ -291,6 +310,17 @@ impl ProviderClient for ProviderOrchestrator {
}
}
});
// `/orphans`: a view over the store, mounted only when the store is.
// Its reference roots are the disk roots of the mounted file providers
// — the `/crabidy` toml tree and (when enabled) the `/fs` root — the
// only places a `Playable::Store` link can live (architecture/orphans.md).
let orphans_client = crabidy_store.as_ref().map(|store| {
let mut ref_roots = vec![store.tree_dir().to_path_buf()];
if let Some(fs) = &fs_client {
ref_roots.push(fs.disk_root().to_path_buf());
}
Arc::new(OrphansProvider::new(Arc::clone(store), ref_roots))
});
// YouTube: non-fatal like the local providers — a missing or
// broken yt-dlp binary only costs the `/youtube` subtree.
let yt_config_file = config_dir.join("ytdy.toml");
@ -316,6 +346,7 @@ impl ProviderClient for ProviderOrchestrator {
fs_client,
crabidy_client,
crabidy_store,
orphans_client,
youtube_client,
})
}
@ -347,6 +378,10 @@ impl ProviderClient for ProviderOrchestrator {
.as_ref()
.is_some_and(|youtube| youtube.is_track_path(path));
}
if orphans_owns(path) {
// Orphans are addressed as nodes; there are no track paths.
return false;
}
false
}
@ -370,6 +405,12 @@ impl ProviderClient for ProviderOrchestrator {
.get_urls_for_track(track_path)
.await;
}
if orphans_owns(track_path) {
return self
.orphans_provider()?
.get_urls_for_track(track_path)
.await;
}
warn!(path = track_path, "no provider owns this track path");
Err(ProviderError::MalformedPath)
}
@ -394,6 +435,12 @@ impl ProviderClient for ProviderOrchestrator {
.get_metadata_for_track(track_path)
.await;
}
if orphans_owns(track_path) {
return self
.orphans_provider()?
.get_metadata_for_track(track_path)
.await;
}
warn!(path = track_path, "no provider owns this track path");
Err(ProviderError::MalformedPath)
}
@ -421,6 +468,14 @@ impl ProviderClient for ProviderOrchestrator {
LibraryNodeChild::new(ytdy::PROVIDER_ROOT.to_owned(), "youtube".to_owned(), false);
root_node.children.push(child);
}
if self.orphans_client.is_some() {
let child = LibraryNodeChild::new(
ORPHANS_PROVIDER_ROOT.to_owned(),
"orphans".to_owned(),
false,
);
root_node.children.push(child);
}
root_node
}
@ -438,6 +493,8 @@ impl ProviderClient for ProviderOrchestrator {
self.crabidy_provider()?.get_lib_node(path).await?
} else if youtube_owns(path) {
self.youtube_provider()?.get_lib_node(path).await?
} else if orphans_owns(path) {
self.orphans_provider()?.get_lib_node(path).await?
} else {
warn!(path, "no provider owns this path");
return Err(ProviderError::MalformedPath);
@ -479,6 +536,12 @@ impl ProviderClient for ProviderOrchestrator {
.create_lib_node(parent_path, title)
.await;
}
if orphans_owns(parent_path) {
return self
.orphans_provider()?
.create_lib_node(parent_path, title)
.await;
}
warn!(parent_path, "no provider supports creating nodes here");
Err(ProviderError::NotSupported)
}
@ -509,6 +572,12 @@ impl ProviderClient for ProviderOrchestrator {
.rename_lib_node(path, new_title)
.await;
}
if orphans_owns(path) {
return self
.orphans_provider()?
.rename_lib_node(path, new_title)
.await;
}
warn!(path, "no provider supports renaming this node");
Err(ProviderError::NotSupported)
}
@ -542,6 +611,12 @@ impl ProviderClient for ProviderOrchestrator {
.resolve_tracks_into(path, chunk_tx)
.await;
}
if orphans_owns(path) {
return self
.orphans_provider()?
.resolve_tracks_into(path, chunk_tx)
.await;
}
warn!(path, "no provider owns this path");
Err(ProviderError::MalformedPath)
}
@ -562,6 +637,9 @@ impl ProviderClient for ProviderOrchestrator {
if youtube_owns(path) {
return self.youtube_provider()?.delete_lib_node(path).await;
}
if orphans_owns(path) {
return self.orphans_provider()?.delete_lib_node(path).await;
}
warn!(path, "no provider supports deleting this node");
Err(ProviderError::NotSupported)
}

3
docs/src/providers.md
View File

@ -66,6 +66,9 @@ present because the server owns it.
queues, bookmarks, and downloaded captures), backed by a
content-addressed store. It has its own page: [The crabidy
store](./store.md).
- **`/orphans`** — store audio that no save references any more, surfaced
for manual reclamation (rename, delete, or queue). See [Reclaiming
orphans](./store.md#reclaiming-orphans--orphans).
Several providers expose a **search** subtree in which you create nodes
whose titles are your search terms; see [Search](./providers/search.md)

29
docs/src/store.md
View File

@ -228,11 +228,34 @@ deleted — another save may still reference it. Because nothing expensive is
ever destroyed, there is no delete-confirmation step.
```admonish note
Nothing reclaims store entries whose last referencing toml was deleted. The
store never shrinks on its own; orphaned entries are accepted, and a garbage
collector is future work.
Nothing reclaims store entries automatically when their last referencing toml
is deleted — the store never shrinks on its own. Those unreferenced entries are
surfaced for manual reclamation by the `/orphans` provider (below).
```
## Reclaiming orphans — `/orphans`
Because deleting a save only removes tomls (never store audio), the store
accumulates entries that nothing references any more. The `/orphans` provider is
the reclamation view: it lists every store entry, walks the mounted local file
providers (the `/crabidy` tree and `/fs`) to cross off the ones a
`Playable::Store` toml still references, and presents the rest.
Each orphan is an ordinary editable/deletable/queueable node, so the usual keys
apply:
- `e` **renames** the store entry — both the audio file and its
`*.cbd-store.toml` sidecar — refusing a name already taken by another entry.
- `d` **deletes** the audio file and its sidecar from disk.
- `a`/`Enter` **queue** it, so you can listen before deciding.
The set is recomputed on every visit (no cached list), so a fresh capture that
re-references an entry makes it drop off the list. "Referenced" means reachable
through a mounted file provider: a `scan --capture` toml under a folder that is
not mounted under `/fs` is not seen, so its target shows here as an orphan. See
`architecture/orphans.md` for the boundary and the (benign) capture-then-delete
race.
## The captured marker
The library marks what you already hold. A captured row ends with a `↓`

7
fsdy/src/lib.rs
View File

@ -531,6 +531,13 @@ impl Client {
&self.provider_root
}
/// The disk root this instance serves. Exposed so the server can hand a
/// file provider's root to the `/orphans` provider as a reference root to
/// scan for `Playable::Store` links (architecture/orphans.md).
pub fn disk_root(&self) -> &Path {
&self.root
}
/// The display title of this instance's root node: the provider root
/// without its leading slash (`fs`, `queues`).
fn root_title(&self) -> &str {

114
plan/orphans.md Normal file
View File

@ -0,0 +1,114 @@
# Task plan — the `orphans` provider
Implements `architecture/orphans.md`, satisfying `quality/orphans.md`. Ordered by
dependency. Each task names its verification. All `cargo`/tooling runs go through
`devenv shell -- …` (see `CLAUDE.md`).
> **Done.** All tasks implemented and verified: `cargo test --workspace`,
> `cargo clippy --workspace --all-targets -- -D warnings`, `cargo fmt --check`,
> and `mdbook build docs` all pass. See "Deviations" at the end.
## Store layer (`crabidy-server/src/crabidy_store.rs`)
- [x] **T1 — `OrphanEntry` + `StoreIndex::remove`.** Move/define `OrphanEntry`
(name + display metadata) and add `StoreIndex::remove(&mut self, name,
sidecar)`, the inverse of `insert` (drop `by_hash[hash]` and each
`by_provider_id[(provider,id)]` that still maps to `name`). *Verify:* unit
test that `insert` then `remove` leaves the index empty; clippy/fmt.
- [x] **T2 — `list_orphans(&self, ref_roots) -> Result<Vec<OrphanEntry>, StoreError>`.**
Scan `store_root` for `*.cbd-store.toml` (name + parsed sidecar); require the
audio file present unless reporting malformed residue (G5). Build the
referenced set by walking each `ref_root` recursively for `*.cbd-track.toml`,
parsing each and collecting `Playable::Store(name)` (skip unreadable/bad files
with a warning). Return all − referenced, metadata from `providers[0]`.
*Verify:* G1–G6 tests.
- [x] **T3 — `orphan_track` / `orphan_url`.** `orphan_track(name)` reads the
sidecar and builds a `Track` (metadata from `providers[0]`, `is_captured =
true`, path left to the provider to set). `orphan_url(name)` returns
`store_root/<name>` as a string, erroring `NoDir`/`MalformedPath`-style if the
entry is gone. *Verify:* G8/G9 tests.
- [x] **T4 — `rename_orphan(&self, old, new)`.** Under the index mutex:
`validate_folder_name(new, &[])`; refuse if `name_taken(new)` (G13); rename
audio and sidecar (`tokio::fs::rename` both, cross-device fallback to
copy+remove like `ingest_file`); read the sidecar, `index.remove(old,
&sidecar)`, `index.insert(new, &sidecar)`. *Verify:* G11–G13 tests.
- [x] **T5 — `delete_orphan(&self, name)`.** Under the index mutex: read the
sidecar (for index removal; tolerate a missing sidecar → still remove any
stray audio), remove audio + sidecar (idempotent on `NotFound`),
`index.remove(name, &sidecar)`. *Verify:* G15–G16 tests.
## Provider (`crabidy-server/src/orphans.rs`)
- [x] **T6 — `entry_name` + leaf/root nodes.** Fill `entry_name` (decode the one
segment; reject bare root, deeper, or separator-bearing paths → `None`).
Implement `get_lib_root` and `get_lib_node`: root lists children from
`store.list_orphans(&self.ref_roots)` (flags per G7, path
`/orphans/<encode(name)>`, title `name`); leaf returns a childless queueable
node with the single `orphan_track` (path set to the leaf path). *Verify:*
G7/G8 tests; `MalformedPath` for unknown/`..` segments (G17).
- [x] **T7 — play/queue + mutations.** Implement `get_urls_for_track`
(`orphan_url`), `get_metadata_for_track` (`orphan_track`), `rename_lib_node`
(validate → `rename_orphan` → return new node), `delete_lib_node`
(`delete_orphan` → return refreshed root). *Verify:* G9, G11–G17 tests via the
provider surface.
## `fsdy` accessor (`fsdy/src/lib.rs`)
- [x] **T8 — `Client::disk_root(&self) -> &Path`.** Public accessor returning the
instance's disk root, so the orchestrator can hand the `/fs` root to the
orphans provider as a reference root. Doc comment (G20). *Verify:* trivial;
compiles + fmt.
## Orchestrator wiring (`crabidy-server/src/provider.rs`, `lib.rs`)
- [x] **T9 — Declare the module.** Add `mod orphans;` (and any `pub use`) to
`crabidy-server/src/lib.rs`; this un-inerts the stub. *Verify:* workspace
compiles.
- [x] **T10 — Mount + route `/orphans`.** Add `orphans_client:
Option<Arc<OrphansProvider>>` to `ProviderOrchestrator`; construct it in
`init` when `crabidy_store` is `Some`, with `ref_roots` = `[store.tree_dir()]`
plus `fs_client.disk_root()` when present. Add `orphans_owns(path)`; route
`is_track_path`, `get_lib_node`, `get_urls_for_track`,
`get_metadata_for_track`, `create/rename/delete_lib_node`, and
`resolve_tracks_into` to it. Add the `orphans` child to `get_lib_root` when
mounted. Note: keep `annotate_captured` running on orphan nodes (already
correct — store-backed tracks are `is_captured`). *Verify:* G21 by reading;
workspace compiles; existing provider tests still pass.
## Docs
- [x] **T11 — mdbook + README.** Add an `/orphans` section to
`docs/src/store.md` (or a short `docs/src/providers/orphans.md` linked from
`SUMMARY.md`) and a line to `README.md`'s provider tree describing `/orphans`
as the store's reclamation view (rename/delete/queue unreferenced audio).
*Verify:* `devenv shell -- mdbook build docs` + markdownlint clean.
## Final gate
- [x] **T12 — Full verification.** `devenv shell -- cargo test --workspace`,
`cargo clippy --workspace -- -D warnings`, `cargo fmt --check`, markdownlint.
Re-read `quality/orphans.md` G1–G21 and check each. Update
`plan/summary.md`. Commit with the co-author trailer.
## Deviations from the plan / architecture
- **Test placement (T2/T4/T5).** Rather than one test per store method, the
provider-level tests in `orphans.rs` exercise `list_orphans`/`orphan_track`/
`orphan_url`/rename/delete end-to-end through the `ProviderClient` surface
(list vs. reference, leaf-track resolution, rename-both-files + delete,
traversal rejection). One store-level test
(`rename_orphan_moves_both_files_and_repoints_the_index`) additionally asserts
G12 (index repoint) by inspecting the private index. All of G1–G21 are
covered; the split just differs from the per-method layout the plan implied.
- **`list_orphans` enumerates a union (G5).** It walks both `*.cbd-store.toml`
sidecars and bare audio files, so a sidecar without audio *and* audio without
a sidecar both surface as reclaimable — as the architecture's "malformed
residue" note requires.
- **Docs home (T11).** The user-facing section landed in `docs/src/store.md`
("Reclaiming orphans — /orphans") with a bullet in `providers.md` and a line
in the README provider tree, rather than a standalone
`docs/src/providers/orphans.md` page — `/orphans` is a view over the store, so
it reads best beside the store doc, and no new `SUMMARY.md` entry was needed.
- **No proto/TUI/web changes (G10 held).** The feature reuses the existing
`is_editable`/`is_deletable`/`is_queable` child handling end-to-end; the only
non-server change is the additive `fsdy::Client::disk_root` accessor.

102
quality/orphans.md Normal file
View File

@ -0,0 +1,102 @@
# Quality gates — the `orphans` provider
Gates for `architecture/orphans.md` and the `crabidy-server/src/orphans.rs`
stub. Each is pass/fail by reading/reasoning or by a named test. Tests live in
`crabidy-server/src/crabidy_store.rs` (the store methods) and
`crabidy-server/src/orphans.rs` (the provider), run via
`devenv shell -- cargo test -p crabidy-server`.
## Correctness of the orphan diff
- [ ] **G1 — All items enumerated.** `list_orphans` counts every store entry
that has both a `<name>.cbd-store.toml` sidecar and a readable `<name>` audio
file. Test: capture two distinct tracks (two store entries), reference
neither → both listed.
- [ ] **G2 — Referenced entries excluded.** An entry referenced by a
`Playable::Store` toml under **any** `ref_root` is not listed. Test: capture
one track into a `/crabidy` save (which writes a store toml), then
`list_orphans` with that tree as a ref root → the entry is **not** an orphan;
delete the save's toml → it **becomes** an orphan.
- [ ] **G3 — `/fs` references count.** A `Playable::Store` toml under the `/fs`
root (as `scan --capture` writes) excludes its target. Test: hand-write a
store toml under an fs ref root → its target is not an orphan.
- [ ] **G4 — References outside ref roots do not count.** A store toml under a
directory that is **not** a ref root does not rescue its target from being an
orphan (documented boundary). Test asserts the target is still listed.
- [ ] **G5 — Malformed residue is reclaimable.** A sidecar with no audio (or
audio with no sidecar) is reported as an orphan, never as "referenced". Test:
drop a lone `x.cbd-store.toml` into the store → listed.
- [ ] **G6 — Fresh every call.** No persisted orphan list; two consecutive
`list_orphans` calls straddling a reference change reflect the change
(covered by G2's two-phase assertion).
## Presentation contract (reuse, no new wire)
- [ ] **G7 — Child capability flags.** Every `/orphans` child advertises
`is_queable`, `is_editable`, `is_deletable`, and `is_captured` all `true`, and
`is_downloadable = false`. The root is `is_queable = true`, `is_creatable =
false`. Verified by reading `get_lib_node`; test asserts the flags on a listed
child.
- [ ] **G8 — Leaf carries exactly one track.** `get_lib_node("/orphans/<seg>")`
returns a childless, queueable node with one track whose `is_captured` is true
and whose metadata matches the sidecar. Test asserts `tracks.len() == 1` and
`children.is_empty()`.
- [ ] **G9 — Queueing resolves the audio.** Resolving an orphan node (default
walk) yields one track whose `get_urls_for_track` returns `store_root/<name>`.
Test drives `resolve_tracks_into("/orphans")` and checks the resolved URL is
the store path.
- [ ] **G10 — No proto/TUI/web change.** Confirm by inspection that the feature
adds no field to any `proto` message and no binding/gesture to `cbd-tui` or
`cbd-web`: it relies solely on existing `is_editable`/`is_deletable`/
`is_queable` handling. (If this gate cannot hold, the design in
`architecture/orphans.md` must be revisited before implementing.)
## Rename safety
- [ ] **G11 — Renames both files.** `rename_orphan(old, new)` renames the audio
`<old>`→`<new>` **and** `<old>.cbd-store.toml`→`<new>.cbd-store.toml`; neither
old name remains on disk. Test.
- [ ] **G12 — Rename updates the index.** After a rename, the index resolves the
same content hash / provider ids to the **new** name and no longer to the old
(so a subsequent capture de-dups against the renamed entry). Test via
`StoreIndex` lookups or a follow-up capture.
- [ ] **G13 — Rename validates and refuses collisions.** An empty / separator /
leading-dot `new` is rejected (`InvalidInput`); a `new` already taken by
another store audio or sidecar is refused without touching disk. Tests for
both.
- [ ] **G14 — Rename is reference-safe by construction.** Only unreferenced
entries are exposed under `/orphans`, so a rename never invalidates a live
toml. Verified by reasoning against the diff definition; no test needed beyond
G2.
## Delete safety
- [ ] **G15 — Deletes both files.** `delete_orphan(name)` removes the audio and
the sidecar and drops the entry from the index. Test asserts both files gone
and `list_orphans` no longer lists it.
- [ ] **G16 — Idempotent.** Deleting an already-gone entry returns `Ok` (a
refreshed root), not an error. Test.
- [ ] **G17 — Delete stays inside the store root.** The name is a validated bare
file name; delete joins it onto `store_root` and never follows separators or
`..`. Verified by reasoning (mirrors `Playable::Store` bare-name validation)
plus a test that a crafted `/orphans/..%2Fx` path is `MalformedPath`, not a
file operation.
## Always-on rules (AGENTS.md)
- [ ] **G18 — No panics on input or I/O.** Every error path (missing entry,
unreadable dir, bad toml, taken rename target, malformed path) is a typed
`ProviderError`/`StoreError`; no `unwrap`/`expect`/`panic!`/`todo!` remains in
shipped code. A bad sidecar or track toml during the walk is skipped with a
warning, never fatal (mirrors `StoreIndex::scan` and `fsdy` listing).
- [ ] **G19 — Bounded work, no unbounded channels.** The reference walk and
enumeration use ordinary async fs iteration; if any channel is introduced it
is bounded. No new external calls (so no timeout/retry surface is added).
- [ ] **G20 — Public items documented.** Every public item in `orphans.rs` and
every new public `CrabidyStore`/`fsdy::Client` method has a doc comment
stating intent and error behavior.
- [ ] **G21 — Mount is optional and non-fatal.** `/orphans` mounts only when the
store is present; its absence (no data/state dir) drops the subtree without
affecting the server, exactly like `/crabidy` and `/fs`. `get_lib_root`
includes the `orphans` child only when mounted. Verified by reading the
orchestrator wiring.