crabidy/plan/soundcloud.md

Task plan — soundcloud provider

Status (2026-07-24): all phases A–E implemented. Everything offline is verified (unit tests, parsers, server build, fmt/clippy/machete). The live-only gates — the real client_id scrape (B2/B3), live JSON shapes (B4/B5), and mp3-HLS play-to-EOS (C4), plus browsing /soundcloud (D4) and tests/live.rs (E1) — need real SoundCloud access and are left to run on a machine with creds and audio. See plan/summary.md.

Executes architecture/soundcloud-provider.md against the stubs in soundclouddy/ and audio-player/src/hls.rs, satisfying the gates in quality/soundcloud-provider.md. Ordered by dependency. Verification column names the test(s) / gate(s) each task must satisfy.

Phase A — Provider logic (no network; unit-testable now)

• A1 — Settings + accessors. Bounds accessors (search_results, playlist_tracks, call_timeout, hls_deadline) with the DEFAULT_* consts; redacting Debug already present. (G1; settings_debug_redacts_secrets.)
• A2 — get_lib_root builds root children directly (sync, no network): search + resolve, plus likes + playlists iff logged_in. Serve the Root node from here (decision #1) and have get_lib_node(Root) delegate to it. (G10; root_without_login_has_no_personal_nodes, root_with_login_shows_personal_nodes.)
• A3 — sc_track mapper ScTrack→proto Track at /soundcloud/track/<id>, provider_item_id = "track:<id>", ms→Option<u32>. (G12.)
• A4 — search_node + term store (create/rename/delete, dedup, implicit recreation). (G17; create_rename_delete_search_term.)
• A5 — search_term_node lists tracks (into node.tracks, canonical paths, queueable node) and playlists (into node.children, canonical container paths, queueable). (G13, G12, G16; search_term_lists_tracks_and_playlists_with_canonical_paths.)
• A6 — resolve_node + URL store (create/rename/delete), and resolve_entry_node: track→node.tracks=[t]; playlist→child container. (G14, G17; resolve_entry_shapes_track_and_playlist, create_resolve_url_entry.)
• A7 — playlist_node fetch detail, hydrate stub ids in ≤50 batches up to playlist_tracks, log truncation. (G15; playlist_node_hydrates_stub_tracks.)
• A8 — likes_node / playlists_node (login only). (G10.)
• A9 — get_urls_for_track / get_metadata_for_track via resolve_stream_url / track_detail; non-track path → MalformedPath. (G18; get_urls_for_track_returns_the_m3u8.)
• A10 — Download blessing in get_lib_node (node + children), and typed fetch_err mapping (no panics). (G4, G16; backend_failures_are_typed.)
• A11 — Gate: cargo test -p soundclouddy — all 15 tests pass. (G11, G22.)

Phase B — HTTP seam ScApi (needs live SoundCloud to fully verify)

• B1 — ScApi::new builds the reqwest client with per-call timeout + user-agent; holds client_id/app_version behind RwLock, optional OAuth. (G3, G6.)
• B2 — scrape_client_id free fn — fetch soundcloud.com, find script bundles, regex client_id + app_version. Unit-test the parser against a captured HTML/JS fixture (no network). (G7; new scrape_parses_fixture test.)
• B3 — Signed GET helper: append client_id/app_version, OAuth header when present; map status→FetchError (404/401/403); re-scrape once on 401/403 then retry. (G8; reauth_rescrapes_once_then_gives_up over a mock/loopback.)
• B4 — Defensive wire DTOs (#[serde(default)], drop id-less entries) and into_* converters for track/playlist/resolve/search/transcoding. (G5.)
• B5 — resolve_stream_url: pick hls+audio/mpeg transcoding, GET its url, return the .m3u8; NotStreamable when absent. (G18.)
• B6 — hydrate_tracks batched /tracks?ids= (≤50). (G15.)
• B7 — settings() folds cached client_id/app_version back in. (G9; settings_round_trip_persists_scraped_id.)
• B8 — init parses TOML, builds ScApi, ensures a client_id (scrape if none), non-fatal on total failure. (G7.)

Phase C — HLS playback in audio-player

• C1 — parse_media_playlist hand-written line parser (#EXTINF + URI, resolve relative against base, one-level master fallback). Unit test with a fixture playlist string. (G19; new hls::tests::parses_media_playlist.)
• C2 — HlsStream state machine (Stream + SourceStream): fetch segments in order, advance at boundaries, finish after last; reconnect re-opens the current segment; forward-only, content_length None, supports_seek false; total-deadline bound. (G6, G19, G20.)
• C3 — Route .m3u8 in open_source to HlsStream (via StreamDownload) before the windowed-HTTP path; opus/other sniffing downstream unchanged. Wire mod hls; (already declared). (G21.)
• C4 — Gate (integration): an ffmpeg-built mp3-HLS fixture (or a live track) plays to EOS with no panic; add a #[ignore] play-to-EOS test. (G19, G20.)

Phase D — Server wiring

• D1 — Workspace dep soundclouddy = { path = "soundclouddy" } in root Cargo.toml [workspace.dependencies]; add to crabidy-server deps.
• D2 — settings.rs: "soundcloud" in ALL_PROVIDERS (→8), ProviderToggles.sc, all(), provider_toggles().
• D3 — provider.rs: sc_client field, sc_owns, sc_provider, build() block reading soundcloud.toml (non-fatal, round-trip settings()), get_lib_root child gated on is_some(), and a routing arm in each dispatch method. (mirrors abs.)
• D4 — Gate: cargo build -p crabidy-server; browse /soundcloud in the running app (manual//run).

Phase E — Validation & docs

• E1 — tests/live.rs (#[ignore]): reads SOUNDCLOUD_OAUTH/optional SOUNDCLOUD_CLIENT_ID from env, exercises scrape→search→resolve→stream_url end-to-end; confirms DTOs match live JSON. (G7 live gate.)
• E2 — Quality sweep: cargo fmt, clippy, machete; tick quality/soundcloud-provider.md. (G23.)
• E3 — plan/summary.md records deviations from this plan.

Notes

• Live-verification boundary: Phases A and C1 are fully verifiable offline. B3/B5/C4/D4/E1 need live SoundCloud access (no creds in CI/sandbox) — deliver as code + fixture/mock tests, run the live gate on a real machine.
• Do not commit secrets; soundcloud.toml stays out of git.