# The crabidy store `/crabidy` is one filesystem provider — the only writable one — where the server keeps everything you save. Its listings look like any other library subtree (see [The library model](./library.md)), but every track toml under it either **links** back to a source provider or **links into a shared, content-addressed store** of audio files. The two halves live in separate directories, split by XDG kind: - the **toml tree** at `~/.local/state/crabidy/` — the folders and `*.cbd-track.toml` files the `/crabidy` provider lists (this is *state*); - the **content store** at `~/.local/share/crabidy/` — a flat directory of audio files, each paired with a sidecar, that captured tomls point at (this is *data*). One writer, `CrabidyStore`, owns both roots and serializes every mutation. The provider only reads: it lists the tomls and resolves store links when a captured track plays. ```admonish note Bookmarks (`w`) and captures (`W`) are two kinds of save in the same tree, told apart by the captured marker — not two separate providers. ``` ## The two roots ```d2 direction: right orchestrator: Library (path-routed) { tidal youtube fs crabidy: "/crabidy (reads tomls,\nresolves store links)" } state: "~/.local/state/crabidy/\ntoml tree" { shape: cylinder } share: { label: "~/.local/share/crabidy/\ncontent store:\naudio + sidecars" shape: cylinder } writer: "CrabidyStore (single writer)" { index: "StoreIndex\n(provider-id -> entry,\nhash -> entry)" } orchestrator.crabidy -> state: lists folders + tomls orchestrator.crabidy -> share: resolves store playables writer -> state: writes save folders + track tomls writer -> share: writes audio + sidecars writer.index -> share: built by scanning sidecars at open ``` Top-level folders under `/crabidy` are **user saves**, each created by `w` or `W`. A saved queue is flat; a saved library subtree keeps its structure (it falls out of walking the source). The reserved `current` folder is the live-queue mirror — see [Queue and playback](./queue.md). ## The store and its sidecars `~/.local/share/crabidy/` is a **flat** directory. Each unique playable is a pair of files: - `` — the audio file, named after the source's natural name: a local file's basename, or a sanitized `.<ext>` for a download (the extension comes from the download's content type or URL). On a name collision with *different* content, a numeral is appended before the extension: `song.flac`, `song (2).flac`, … Identical content never reaches naming — it de-duplicates first. - `<name>.cbd-store.toml` — the sidecar: the entry's content hash and every provider identity that maps to it. ```toml # song.flac.cbd-store.toml hash = "blake3:1f0c…" # content hash of the audio file [[provider]] # one entry per provider identity provider = "tidal" # provider name (source path root) id = "125169484" # provider-internal id title = "Bohemian Rhapsody" artist = "Queen" duration = 355 aliases = ["Bohemian Rhapsody (Remastered)"] # other titles for this id [[provider]] # same audio reached via a second identity provider = "youtube" id = "fJ9rUzIMcZQ" title = "Queen – Bohemian Rhapsody (Official Video)" ``` The **sidecars are the single source of truth** — there is no separate persisted index. At open, `CrabidyStore` scans every `*.cbd-store.toml` and builds an in-memory `StoreIndex`: - `by_provider_id: (provider, id) -> store name` - `by_hash: content hash -> store name` Both lookups are O(1), and the index is updated on every write. A malformed sidecar is skipped with a warning, so one bad file never poisons the index. ```admonish tip The store name is opaque — only the toml's title shows in the UI. Two saves that reference the same audio point their tomls at the same `<name>`, so capturing a track twice from different places costs one copy on disk. ``` ### Provider identity The store keys on a **provider-internal id**, not on a path (the same item is reachable through a playlist, a search, an album…). Each provider sets it on the tracks it produces: - **tidal** → the numeric track id - **youtube** → the video id - **fs** → the source file's absolute path (two `/fs` tomls pointing at one file share an id, so they de-duplicate) An empty id means the provider exposes none; identity then falls back to the content hash alone. ## Saving: `w` and `W` A save takes a *source* — a live-queue snapshot or a library-node path — and writes a new top-level folder `/crabidy/<name>`: - **`w` (bookmark).** Writes a folder of **link** tomls, one per track, each carrying the entry's metadata and a `link` playable back to the source. No audio, no store. Reloading rewrites each link back to its target. - **`W` (capture).** Writes the same folder, but each track's audio is fetched into the content store and its toml carries a `store` playable. Works on both a library node and the queue. ```admonish warning A download capture (`W`) can take a long time — it fetches every track's audio. De-duplication (below) makes a re-capture cheap, but the first one is bounded only by the source size and the byte budget. ``` ### Atomic saves, no overwrite Every save is built in a hidden `.tmp-<name>` sibling folder and renamed into place on success; a failed run removes the temp folder and leaves nothing behind. If `/crabidy/<name>` **already exists**, the save is refused with `name "<name>" already exists` — nothing is clobbered. To replace a save, you delete the old folder first. Resumability comes from the **store**, not the folder: a failed capture leaves the name free to retry, while any audio already committed to the store persists and makes the retry fast (the de-dup flow reuses it). The folder itself is all-or-nothing. `current` is exempt — the playback loop overwrites it on every queue change, and you cannot save over that reserved name. ## De-duplication on capture For each source track, a capture walks these steps in order and writes exactly one toml at the end: ```d2 direction: down start: "resolve source track\n(provider, id, natural name)" already: "already store-backed?\n(local file under the store root)" byid: "index.by_provider_id[(provider, id)] ?" getbytes: "obtain bytes\n(download to temp, or local file)" byhash: "index.by_hash[hash(bytes)] ?" newentry: "NEW: copy into store (+numeral)\nand write sidecar" addid: "add identity to sidecar,\ndiscard the temp copy" writetoml: "write track toml with a store playable" start -> already already -> writetoml: "yes: reuse the target name (no copy)" already -> byid: "no" byid -> writetoml: "HIT: reuse; record a differing title as an alias" byid -> getbytes: "MISS" getbytes -> byhash byhash -> addid: "HIT (same content, new identity)" byhash -> newentry: "MISS" addid -> writetoml newentry -> writetoml ``` 1. **Already store-backed?** If the source resolves to a local file already inside the store root, there is nothing to fetch — the toml links to that same `<name>`. (Capturing an already-captured item does nothing.) 2. **Provider-id lookup.** If `(provider, id)` is already in the index, reuse that entry and skip the download. If the current title differs from the stored one, it is appended to that identity's `aliases`. This is the common re-capture path. 3. **Miss → obtain bytes.** A streamed source is downloaded to a temp file; a local source *is* the bytes. The bytes are hashed with blake3. 4. **Hash lookup.** If the hash is already in the index, identical content is already stored under some other identity: a new `[[provider]]` entry is added to that sidecar, the temp download is discarded, and the toml points at the existing entry. No duplicate. 5. **Miss → new store entry.** A store name is chosen from the natural name (with a numeral on collision), the temp file is moved (or the local file copied) into the store, and the sidecar is written with the hash and the first identity. The byte budget counts only bytes fetched *this run*, so de-dup makes a big save cheaper and never starves it. Local `/fs` files are always **copied** — the original in your music folder stays put — and de-duplicated by hash like anything else. ## Skipped tracks A source that genuinely cannot be captured — a track already marked skipped, a stream that fails to resolve, a missing or unreadable local file — is recorded as a **skipped** toml (`[playable] skipped = true`) rather than silently omitted, so the save's track list matches the source. Skipped tracks render red in the TUI (see [The terminal UI](./clients/tui.md)) and playback skips over them (see [Queue and playback](./queue.md)). ## Deletion Deletion on `/crabidy` goes through directly, **with no confirmation, and never touches the store**: - Delete a track → remove its `.cbd-track.toml` only. - Delete a folder → remove that toml folder only. A `store` playable resolves under `~/.local/share/…`, which is outside the `/crabidy` toml root at `~/.local/state/…`, so the shared audio is never deleted — another save may still reference it. Because nothing expensive is ever destroyed, there is no delete-confirmation step. ```admonish note Nothing reclaims store entries automatically when their last referencing toml is deleted — the store never shrinks on its own. Those unreferenced entries are surfaced for manual reclamation by the `/orphans` provider (below). ``` ## Reclaiming orphans — `/orphans` Because deleting a save only removes tomls (never store audio), the store accumulates entries that nothing references any more. The `/orphans` provider is the reclamation view: it lists every store entry, walks the mounted local file providers (the `/crabidy` tree and `/fs`) to cross off the ones a `Playable::Store` toml still references, and presents the rest. Each orphan is an ordinary editable/deletable/queueable node, so the usual keys apply: - `e` **renames** the store entry — both the audio file and its `*.cbd-store.toml` sidecar — refusing a name already taken by another entry. - `d` **deletes** the audio file and its sidecar from disk. - `a`/`Enter` **queue** it, so you can listen before deciding. The set is recomputed on every visit (no cached list), so a fresh capture that re-references an entry makes it drop off the list. "Referenced" means reachable through a mounted file provider: a `scan --capture` toml under a folder that is not mounted under `/fs` is not seen, so its target shows here as an orphan. ```admonish note The list is a snapshot taken when you open `/orphans`. A capture that starts right after can adopt an entry you are still looking at, so deleting a just-listed orphan can in principle remove audio a capture has meanwhile claimed — the capture then re-downloads it. Nothing is corrupted; revisit the node for a fresh list. ``` ## The captured marker The library marks what you already hold. A captured row ends with a `↓` (down-arrow) — a trailing status marker after any action-key brackets (`[e]`/`[d]`), e.g. `Bohemian… ↓`: - A **track** is captured when its playable is store-backed, or when its `(provider, id)` is in the store index. Because the check is one index lookup, it works **while browsing any provider** — you can see, in `/tidal`, which tracks you have already captured. - A **node** is captured when all of its tracks are captured and it has no child nodes. ## See also - [The library model](./library.md) — nodes, tracks, paths, and links. - [Providers](./providers.md) — how each media source sets its identity. - [Queue and playback](./queue.md) — the `current` mirror and skipped tracks. - [Configuration](./config.md) — where the two roots come from.