crabidy/architecture/progressive-queueing.md

Progressive queueing of large collections

Context and problem statement

Queueing a nested node (an artist with many albums, a large playlist) today freezes the UI's mental model: nothing changes for many seconds, then the full queue appears at once. Three compounding causes, all in the resolve path:

1. Playback::resolve_tracks collects every track before the queue is touched, so the single Queue broadcast happens only at the very end and playback cannot start earlier.
2. The resolve runs inline in the playback loop, so every other playback command — pause, next, volume — is blocked for the duration.
3. tidaldy paginates collections to exhaustion (50 tracks per sequential request) inside one get_lib_node call, so even a single large playlist produces no intermediate result.

The feature: resolve progressively. Tracks are applied to the queue in chunks as the provider produces them, each chunk is broadcast, playback starts with the first chunk, and clients see a loading indicator (animated dots as a pseudo last queue item) while resolution is still running.

Run autonomously per standing user instruction; every decision below records the options considered and the rationale.

Assumptions

• Chunks must arrive in playback order — the user explicitly wants "the first elements first, and later chunks follow". Order is known up front (collection order), so no reordering step is needed.
• The four resolve-based queue operations (Replace, Queue, Append, Insert) all benefit equally and should share one mechanism.
• Wire changes must stay additive (old clients keep working; they simply see the queue fill progressively without an indicator).
• Multi-client remains supported: the indicator must be server-derived state, not client-local guessing.

Decisions

D1 — End-to-end progressive resolution, not a client-only spinner

Options considered:

1. Client-only indicator: the TUI shows dots between sending a queue op and receiving the next Queue update. No wire or server change.
2. Server-side chunked resolution with a wire-visible "still resolving" flag; the indicator falls out of the flag.

Decision: (2). Option 1 papers over the latency without fixing it — playback would still start only after the full resolve, other clients would see nothing, and the "did it even register?" dead time remains. Option 2 fixes the actual complaint (start playing early, fill visibly) and gives every client the indicator for free. The server broadcasts an immediate Queue update (unchanged tracks, resolving = true) when the op is accepted, so feedback appears within one round trip.

D2 — Queue.resolving field, not a new stream-update variant

Options considered:

1. New GetUpdateStream oneof variant resolving(bool).
2. New field bool resolving = 4 on the Queue message itself.

Decision: (2). The flag is queue state and must be atomic with the track snapshot it describes; a separate variant can arrive out of order relative to Queue updates (the broadcast channel is lossy for slow clients). Additive field, wire-compatible both ways: old clients ignore it, old servers never set it.

D3 — Chunked resolution lives in ProviderClient, with a default

Options considered:

1. Keep the BFS in ProviderOrchestrator (chunk = one node's tracks); no trait change. A 1000-track playlist is still one 20-request blob.
2. Add a chunked resolve method to the ProviderClient trait with a default implementation (the generic walk, one chunk per node); tidaldy overrides it to stream page-sized chunks (50) for the paginated collections (playlists, albums).

Decision: (2).

/// Streams the playable tracks under `path` into `chunk_tx` in playback
/// order. Zero or more chunks, then the sender is dropped: a dropped
/// SENDER means resolution finished. A dropped RECEIVER cancels
/// resolution (the provider stops fetching and returns Ok).
async fn resolve_tracks_into(
    &self,
    path: &str,
    chunk_tx: flume::Sender<Vec<Track>>,
) -> Result<(), ProviderError>;

The channel semantics are the contract and are documented on the trait — this is a local bounded channel used as a stream, not a queue pretending to be durable. Unreadable nodes are skipped with a warning (today's behavior); only a completely unresolvable root returns an error. The default implementation walks the tree depth-first pre-order over queueable descendants, emitting one chunk per node. This replaces the old orchestrator BFS, and fixes a latent ordering bug while doing so: the old worklist popped LIFO, so an artist's albums were flattened in reverse order.

tidaldy overrides the method: playlist and album paths stream one chunk per fetched page instead of paginating to exhaustion first (a new make_paginated_request variant hands each page to a sink); everything else follows the generic walk. Search-term track lists are a single page already. The playlist arm also stops fetching the playlist metadata (title) — the resolve needs only tracks.

flume is already a workspace dependency; crabidy-core adopts it for the trait signature.

D4 — Neither event loop blocks: spawned resolves, single-writer queue

Options considered:

1. Consume chunks inline in the playback loop's handle_command (loop still blocked for the whole resolve; pause/next dead — today's hidden defect).
2. Spawn the resolve; queue mutations travel back to the playback loop as internal commands, so the loop stays the single writer of queue state.

Decision: (2), on both loops.

• Provider side: ProviderOrchestrator::run wraps the orchestrator in an Arc; the ResolveTracks arm spawns the resolve onto its own task instead of awaiting it inline. Without this, the first chunk would deadlock the system: playback applies chunk 1 → play() → sends GetTrackUrls and awaits the reply — but the provider loop would still be busy resolving. Other provider commands keep flowing while (possibly several) resolves run.
• Playback side: each queue op registers a pending op (id from an AtomicU64, kind, insertion cursor) and spawns a forwarder task that drives ProviderCommand::ResolveTracks per path (concurrently, under the exponential read-ahead window of D8, but forwarding chunks in strict multi-path order) and forwards each chunk to the playback channel as PlaybackCommand::ApplyResolvedChunk { op_id, tracks }, followed by ResolveFinished { op_id }. Queue state is only ever mutated inside the loop, exactly as before; user commands interleave between chunks.

Backpressure is real at every hop: provider → forwarder over a small bounded chunk channel, forwarder → playback over the existing bounded(64) command channel. A slow consumer slows the HTTP fetching down instead of buffering unboundedly.

D5 — Chunk application semantics per op kind

Each pending op keeps an insertion cursor:

• Replace: first chunk replace_with_tracks (broadcast resets the queue, current position 0, playback starts); later chunks append.
• Append: every chunk append_tracks.
• Queue (after current): cursor starts at the current position; each chunk insert_tracks(cursor), then cursor += chunk.len().
• Insert: same, starting at the requested position.

Playback start reuses the existing Option<Track> returns from the QueueManager mutations — only a chunk that makes a track current (replace, or any insert into an empty queue) yields one, so exactly the first relevant chunk starts the player and later chunks never restart a playing track.

That first start can fail to find anything playable yet: if the head tracks are is_skipped or the collection's first track is very short, the player can run dry before the next chunk resolves — and then, since later chunks return None, playback would stay stopped with playable tracks arriving right behind it. So the op carries a wants_start flag: set when a chunk makes a track current, cleared only once a start is confirmed (play returned that it handed a track to the player). While it is set, each arriving chunk retries the start from the current position — next_playable advances past skipped/unplayable heads to the first track that has now resolved. The exponential read-ahead (D8) makes that next track arrive sooner; the retry makes sure it actually plays when it does. An op whose whole resolve finishes with nothing playable is dropped by finish_resolve and the player simply stays stopped.

Interleaved edits from other clients during a resolve can shift the cursor's target (e.g. removing tracks before it). This is accepted as benign: insert_tracks already clamps, the queue self-heals on the next broadcast, and simultaneous multi-client edits during a resolve are rare. Under shuffle, arriving chunks are shuffled behind the current track like any other insert — chunk order is irrelevant when shuffle is on. Each op counts its applied tracks; an op that finishes with zero keeps today's "resolved to no playable tracks" warning.

D6 — Replace and Clear cancel in-flight resolves

Without cancellation, "replace the queue" or "clear the queue" during a large resolve would be followed by the old op's remaining chunks trickling back in — a corrupted queue, and the exact ghost behavior this feature is meant to kill. Options: let stale chunks land (wrong), or cancel.

Decision: cancel. Each pending op carries an Arc<AtomicBool> shared with its forwarder. Replace and Clear mark every pending op cancelled and drop it from the map. The forwarder checks the flag per chunk and, when set, drops the chunk receiver — the provider's next send fails and the resolve task stops fetching (the documented receiver-drop semantics from D3). Chunks already in flight for an unknown op id are ignored by the loop. Queue/Append/Insert do not cancel: concurrent additive ops are legal; their chunks interleave between ops while each op's internal order is preserved.

D7 — TUI indicator: an animated pseudo-item, outside the list model

While the latest Queue update carries resolving = true, the queue pane renders one extra line after the last track: one to three dots cycling (~400 ms per step, derived from elapsed time — the render loop already redraws at least every 100 ms), in COLOR_SECONDARY. The pseudo-item is appended at render time only and never enters self.list, so selection, removal, and get_size cannot reach it — no new input states, nothing to misclick.

D8 — Exponential read-ahead across paths

The forwarder of D4 first resolved paths one at a time. That starts the first track quickly (good) but fills the rest only as fast as one provider resolve at a time. When enumeration is slow (per-path Tidal/YouTube/fyyd round trips) and the leading tracks are very short or is_skipped, playback drains the resolved queue faster than a sequential resolver refills it — and stalls into silence, the exact thing progressive queueing exists to avoid.

Options considered:

1. Sequential per path (original): simplest, but a short/skipped head can outrun a slow resolver.
2. Resolve every path at once: fills fastest, but a large marked selection fires an unbounded burst of concurrent provider calls (rate limits, memory) the instant playback starts — wasteful when the user skips away after two tracks.
3. Exponential read-ahead window: a concurrency window that starts at 1 and doubles (1, 2, 4, 8, 16, then steady 16) after each path completes.

Decision: (3). The first path resolves alone, so time-to-first-track is unchanged from (1); the window then grows geometrically, so the resolved queue runs exponentially ahead of linear playback and a short/skipped head cannot catch it after the first couple of tracks. The cap (16) bounds concurrent provider load. Chunks are still forwarded in strict path order — the forwarder fully drains the oldest in-flight resolve before the next, so concurrency never reorders the queue (D5's per-op cursor and "first chunk starts the player" are untouched). Cancellation (D6) drops the in-flight receivers, stopping every concurrent resolve at once.

This is a read-ahead over paths. A single collection path (one album or playlist) is still enumerated by its provider's resolve_tracks_into — the per-page streaming of D3 is that path's read-ahead — so the window is the win for multi-item selections; single-collection latency stays a provider concern.

Flows

shape: sequence_diagram
user: { shape: person }
tui: cbd-tui
rpc: gRPC handler
playback: playback loop
fwd: forwarder task
provider: provider loop
resolve: resolve task
tidal: Tidal API

user -> tui: queue large artist
tui -> rpc: Append(paths)
rpc -> playback: "PlaybackCommand::Append (fire-and-forget)"
playback -> playback: register pending op
playback -> tui: "Queue update (resolving=true)"
playback -> fwd: spawn
fwd -> provider: "ResolveTracks(path, chunk_tx)"
provider -> resolve: spawn
resolve -> tidal: fetch page 1
resolve -> fwd: chunk 1
fwd -> playback: "ApplyResolvedChunk(op, chunk 1)"
playback -> tui: "Queue update (resolving=true)"
playback -> playback: "play() first track"
resolve -> tidal: fetch page 2
resolve -> fwd: chunk 2
fwd -> playback: "ApplyResolvedChunk(op, chunk 2)"
playback -> tui: "Queue update (resolving=true)"
fwd -> playback: "ResolveFinished(op)"
playback -> tui: "Queue update (resolving=false)"

The dots pseudo-item is visible in the TUI exactly while updates carry resolving = true; user commands (pause, next, remove) flow through the playback loop between chunk applications instead of waiting for the end.

direction: right
core: "ProviderClient::resolve_tracks_into" {
  default: "default: pre-order walk,\none chunk per node"
}
tidaldy: "tidaldy override" {
  pages: "playlist/album:\none chunk per 50-track page"
}
playback: "playback loop" {
  ops: "pending ops:\ncursor + cancel flag"
}
core -> tidaldy: overridden by
tidaldy.pages -> playback.ops: "bounded chunks, in order"
playback.ops -> playback.ops: "apply + broadcast per chunk"

Boundaries and risks

• Proto: one additive field (Queue.resolving = 4). No RPC shape changes; the queue ops stay fire-and-forget.
• Trait: one new ProviderClient method with a default implementation — existing providers (there is one) compile unchanged if they skip the override; the override is where the provider-level win lives.
• Ordering fix is a behavior change: multi-album artists now queue in listing order instead of reversed. Strictly a fix, noted here because someone may have gotten used to the bug.
• Concurrent additive ops interleave between ops. Each op's internal order is kept; the interleaving matches command arrival order at the loop. Accepted — same semantics a human doing two appends "at once" expects.
• Old TUI + new server: queue fills progressively, no indicator — pure improvement, no breakage. New TUI + old server: resolving is always false, indicator never shows, behavior as today.
• Not in scope: pagination of get_lib_node for browsing (the library pane still fetches collections to exhaustion before rendering), queue persistence, a progress percentage (total counts are known per collection but not aggregated across a nested walk).