crabidy/architecture/queue-persistence.md

Queue persistence

Context and problem statement

The queue lives only in the playback loop's memory: restarting crabidy-server loses it. The user wants

1. an automatically maintained current queue, persisted on every queue operation and reloaded when the server starts, and
2. named saved queues: pressing w on the queue pane asks for a name and stores the current entries under it.

The explicit framing: realize this completely with the fs provider — a second fsdy instance pointed at a queues/ folder inside the crabidy config directory, one subfolder per queue, each holding serialized track files.

Assumptions (confirmed against the code)

• The proto already declares SaveQueue(SaveQueueRequest{name}); the server handler is a no-op stub (rpc.rs). No wire change is needed.
• The playback loop is the single writer of queue state (Playback.queue: Mutex<QueueManager>); every content change funnels through broadcast_queue, every current-track change through play (plus the shuffle/repeat toggles). Hooking those sites observes every queue operation.
• fsdy track files carry metadata plus one playable; a link playable rewrites Track.path to its target at listing time (architecture/fs-provider.md D2). Queueing a folder of link files therefore reconstructs the original tracks with zero new mechanisms.
• w is unbound in the TUI's Queue scope; the input overlay (InputState/InputPurpose) already handles ask-for-a-name flows.

Decisions

D1 — Mount a second fsdy instance at /queues

Options considered:

• (a) A new provider crate (queuedy) that owns the queues folder.
• (b) Parameterize fsdy::Client with its provider root and mount a second instance at /queues over <config>/crabidy/queues.

Decision: (b) — the user's framing, and the listing/parsing/routing logic is byte-for-byte the same. fsdy::Client gains a constructor Client::new(provider_root, disk_root); the ProviderClient::init path keeps building the /fs instance from fsdy.toml. The hardcoded "/fs/" prefixes in disk_path/is_track_path/list_dir become instance state. The orchestrator gains queues_client and /queues routing arms; init creates the folder (create_dir_all) and is non-fatal like /fs (a failure costs persistence, never the server). Loading a saved queue is just browsing /queues and queueing a folder — no new RPCs, no new TUI flows.

D2 — Persist a queue as a folder of order-prefixed link files

Every queue entry becomes NNNN <title>.cbd-track.toml with the entry's metadata (title, artist, duration, album) and playable.link = Track.path — uniformly, for every entry. The 4-digit zero-padded prefix makes the case-insensitive listing sort reproduce queue order; the sanitized title keeps the files human-readable. Round trip: listing rewrites each link track's path back to its target, so reloading yields the original tracks with the persisted metadata.

Consequence — the "no links into /fs" rule falls. A queue may contain /fs/... tracks (file/url playables keep their fs path), so persisted files must be able to link into an fs-provider instance. The original rejection (fs-provider D3) existed to prevent chains; it is replaced by the stronger structural argument: links are one hop by construction — get_urls_for_track never follows a link (a link-playable target is MalformedPath), so a link whose target is itself a link file dies at play time with a warning, and cycles cannot recurse anywhere. TrackFileError::LinkIntoFs is removed; a link must merely be an absolute path. architecture/fs-provider.md D2/D3 are reconciled with this.

Not chosen: inlining the target's file/url playable into the saved file — the persister only has the wire Track (path + metadata), and links keep the saved queue pointing at the node, surviving edits to the underlying track file.

D3 — Layout: <config>/crabidy/queues/<name>/, current queue = current

• The automatically maintained queue lives in queues/current/ — a visible, ordinary queue folder (it shows up under /queues like any saved queue). The name is reserved: SaveQueue("current") is rejected so a named save is never silently clobbered by auto-persist.
• Each queue folder carries a hidden sidecar .queue-state.toml (current_position, repeat, shuffle). Dot-prefixed → invisible to the provider listing. It is written for every queue and read only when restoring current at startup.
• Writes go to a hidden sibling temp dir (.tmp-<name>), then the old folder is removed and the temp renamed into place. Not atomic (rename over a non-empty dir is impossible); the crash window can lose the folder — accepted for a local music queue, and a warning covers it.
• Saving an existing name overwrites it (same temp-and-swap).

D4 — Auto-persist through a latest-wins channel and one persister task

The playback loop must never block on disk. Every queue-state change sends a snapshot (tracks, current_position, repeat, shuffle) into a tokio::sync::watch channel (bounded, single slot, latest wins — a burst of resolve chunks coalesces naturally). A dedicated persister task awaits changes, debounces briefly, skips writes whose snapshot equals the last one written (broadcasts that only toggled the resolving flag stay free), and rewrites queues/current/ per D3. Disk failures are warnings; playback is never affected. Send sites: the broadcast_queue funnel, the current-track broadcast in play, and the shuffle/repeat toggle handlers.

D5 — Restore at startup, bespoke, never autoplay

Replace(["/queues/current"]) through the normal resolve flow was rejected: it starts playback (a restarted server must stay silent), and it cannot restore the queue position. Instead, before the loops start serving, the server reads queues/current/ directly — sorted listing, TrackFile::parse, to_track (identical semantics to the provider) — applies the tracks to the QueueManager, restores current_position/repeat/shuffle from the sidecar, and leaves PlayState::Stopped. A missing folder is a fresh start; a broken file is skipped with a warning like any listing.

D6 — SaveQueue wiring

rpc save_queue sends PlaybackCommand::SaveQueue { name, result_tx } to the playback loop (single-writer discipline: only the loop may snapshot). The loop validates and snapshots, then hands the write to a spawned task so it never blocks on disk; the RPC reply reports the actual write result. Errors: invalid name (empty after trim, contains a path separator or NUL, starts with ., or is current) → invalid_argument; empty queue → failed_precondition; I/O → internal.

D7 — TUI: w on the queue pane

New Action::QueueSaveAs bound to w in Scope::Queue ("Save queue as…"). It opens the existing input overlay with a new InputPurpose::SaveQueue (label save queue), no-op while the queue is empty. Submit sends MessageFromUi::SaveQueue(name) → the SaveQueue RPC. The saved queue appears under /queues on the next library visit — no push update needed.

D8 — Out of scope (explicitly)

• Renaming/deleting saved queues from the TUI (fsdy keeps create/rename/delete NotSupported); file tools work today, marking /queues children deletable is future work.
• Making the queues directory configurable; it is derived from the config dir.
• Persisting the playback position within the track, autoplay on restore, or multiple current queues.

Structure

direction: right

server: crabidy-server {
  pb: Playback loop {
    q: "QueueManager (single writer)"
  }
  persister: "persister task" {
    w: "debounce, skip unchanged,\nwrite current/"
  }
  store: QueueStore {
    s: "validate name, tmp-and-swap"
  }
  orch: ProviderOrchestrator
}

fs: "fsdy /fs\n(music root)"
qfs: "fsdy /queues\n(config queues dir)"

disk: "config/crabidy/queues" {
  shape: cylinder
  cur: "current/ + .queue-state.toml"
  saved: "<name>/ per saved queue"
}

server.pb -> server.persister: "watch channel\n(latest snapshot wins)"
server.persister -> server.store: persist current
server.pb -> server.store: "SaveQueue(name)\n(spawned write)"
server.store -> disk
server.orch -> qfs: "/queues/..."
server.orch -> fs: "/fs/..."
qfs -> disk: "list + parse (read only)"

Key flow: save, restart, reload

shape: sequence_diagram
tui: TUI
rpc: gRPC
pb: Playback loop
store: QueueStore
orch: Orchestrator

tui -> rpc: "SaveQueue(road trip)"
rpc -> pb: "PlaybackCommand::SaveQueue"
pb -> store: "snapshot -> spawned write"
store -> rpc: "queues/road trip/ written"
rpc -> tui: OK
tui -> pb: "(server restarts; restore reads current/)"
tui -> orch: "GetLibraryNode(/queues)"
orch -> tui: "children: [current, road trip]"
tui -> pb: "ReplaceQueue([/queues/road%20trip])"
pb -> orch: "resolve: links rewritten to targets"

Risks and open questions

• Hand-written files in queues/ behave like any fs tree (broken files skipped with warnings). A hand-written url/file track keeps its /queues/... path when queued; persisting then links to that file — one hop, resolves fine.
• Queues past 9999 tracks sort wrong beyond the 4-digit prefix; accepted (prefix width is a constant).
• Concurrent saves to the same name race on the temp dir; last writer wins. Accepted for a single-user local server.
• Metadata drift: a saved queue replays the metadata captured at save time, not the target's live metadata — consistent with fs-provider D2.
• Open (future): deletable saved queues in the TUI; a SaveQueue confirmation/overwrite prompt; persisting the in-track position.