Add the SoundCloud provider (`/soundcloud`) with HLS playback
New `soundclouddy` crate mounted at `/soundcloud`: search tracks and playlists, resolve permalink URLs, and — with an optional OAuth token — the user's likes and playlists (public browse/play needs only a client_id). Ran the full dev-flow: architecture/soundcloud-provider.md, quality/soundcloud-provider.md, plan/soundcloud.md, plan/summary.md. - Provider logic over an `Sc` reqwest seam (faked in tests): creatable `search`/`resolve` parents, canonical `track/<id>` and `playlist/<id>` leaves, playlist hydration, download blessing — mirrors abs/fyyd. - Auth: `client_id` from config or scraped from soundcloud.com (pure parsers, unit-tested), re-scraped once on 401; scraped id persisted via `settings()`. - Playback: a new `HlsStream` SourceStream in audio-player streams the m3u8's mp3 segments in order as one continuous mp3; `open_source` routes `.m3u8` to it, non-seekable so symphonia never end-seeks a length-less stream. - Wired into crabidy-server the standard way (settings toggle, sc_owns/ sc_provider, non-fatal build block, root child, dispatch arms). Verified offline: soundclouddy 19 tests, audio-player 14 (incl. HLS-parser), crabidy-server 77+4 — all green; fmt/clippy/machete clean. The live client_id scrape, real JSON shapes, and mp3-HLS play-to-EOS need real SoundCloud access and are covered by tests/live.rs + #[ignore] gates (quality G7/G8/G14/G19). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Test User
parent
9fba9708bf
commit
7091d37c32
|
|
@ -1173,6 +1173,7 @@ dependencies = [
|
|||
"realfft",
|
||||
"reqwest 0.13.1",
|
||||
"serde",
|
||||
"soundclouddy",
|
||||
"tempfile",
|
||||
"thiserror 2.0.19",
|
||||
"tidaldy",
|
||||
|
|
@ -4771,6 +4772,20 @@ dependencies = [
|
|||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "soundclouddy"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"crabidy-core",
|
||||
"reqwest 0.13.1",
|
||||
"serde",
|
||||
"thiserror 2.0.19",
|
||||
"tokio",
|
||||
"toml",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "spin"
|
||||
version = "0.9.9"
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ members = [
|
|||
"crabidy-server",
|
||||
"fsdy",
|
||||
"fyyd",
|
||||
"soundclouddy",
|
||||
"tidaldy",
|
||||
"ytdy",
|
||||
]
|
||||
|
|
@ -108,5 +109,6 @@ crabidy-core = { path = "crabidy-core" }
|
|||
crabidy-server = { path = "crabidy-server" }
|
||||
fsdy = { path = "fsdy" }
|
||||
fyyd = { path = "fyyd" }
|
||||
soundclouddy = { path = "soundclouddy" }
|
||||
tidaldy = { path = "tidaldy" }
|
||||
ytdy = { path = "ytdy" }
|
||||
|
|
|
|||
|
|
@ -0,0 +1,292 @@
|
|||
# soundcloud provider (streaming music)
|
||||
|
||||
## Context and problem statement
|
||||
|
||||
A new library provider mounted at `/soundcloud` that lets a user **search,
|
||||
resolve share links, and play** SoundCloud tracks and playlists — and, **when
|
||||
they opt in with a token, browse their own likes and playlists**.
|
||||
|
||||
- Like fyyd/tidal/youtube, SoundCloud is a **remote, search-driven** service.
|
||||
Unlike them, SoundCloud offers **no official public API**: the modern
|
||||
`api-v2.soundcloud.com` requires a `client_id` that SoundCloud embeds in its
|
||||
web app and **rotates periodically**, and personal-account access needs an
|
||||
**OAuth token**. So the provider must (a) obtain a `client_id` on its own and
|
||||
survive rotation, and (b) treat login as **optional** — public browse + play
|
||||
works with only a `client_id`; a token merely adds personal nodes.
|
||||
- Content is organized as **tracks** and **playlists** (a playlist is a
|
||||
container of tracks). There is no per-track container level like abs books —
|
||||
the tree is `search-term → tracks`, `playlist → tracks`, and (logged in)
|
||||
`likes → tracks` / `playlists → playlist → tracks`, plus a **resolve** entry
|
||||
that turns a pasted permalink URL into a track or playlist.
|
||||
- **Playing** a track is the biggest divergence from every existing provider.
|
||||
SoundCloud does not serve a plain file URL: each track carries a set of
|
||||
`media.transcodings`, and the playable ones are **HLS** — an `.m3u8` playlist
|
||||
of short **mp3 segments**. crabidy's player streams a single byte source, so
|
||||
this requires a new **HLS source** in `audio-player` that fetches the playlist
|
||||
and streams the mp3 segments in order as one continuous mp3 (mp3 frames
|
||||
byte-concatenate into a valid stream — the same fact `ffmpeg -c copy` relies
|
||||
on). rodio's existing symphonia mp3 path then decodes it, unchanged.
|
||||
- **Captures** (`W`, download) come for free once nodes serve tracks and raise
|
||||
`is_downloadable`, exactly as for abs/fyyd — no wire or TUI work.
|
||||
|
||||
## The SoundCloud API (grounding)
|
||||
|
||||
Base `https://api-v2.soundcloud.com`. **Every** request carries `?client_id`
|
||||
(plus `app_version`, `app_locale=en`), omitted from the cells below; personal
|
||||
calls also send `Authorization: OAuth <token>`. `search/*` add
|
||||
`&limit=<n>&offset=<o>&linked_partitioning=1`. Endpoints we use:
|
||||
|
||||
| Purpose | Endpoint |
|
||||
| --- | --- |
|
||||
| Resolve a permalink URL | `GET /resolve?url=<permalink>` |
|
||||
| Search tracks | `GET /search/tracks?q=<t>` |
|
||||
| Search playlists | `GET /search/playlists?q=<t>` |
|
||||
| Track detail | `GET /tracks/<id>` |
|
||||
| Playlist detail | `GET /playlists/<id>` |
|
||||
| Transcoding → media URL | `GET <transcoding.url>` → `{"url": "<m3u8>"}` |
|
||||
| (login) My likes | `GET /me/likes/tracks` (OAuth) |
|
||||
| (login) My playlists | `GET /me/playlists` (OAuth) |
|
||||
|
||||
Objects (fields we read):
|
||||
|
||||
- **track**: `id` (numeric), `title`, `user.username` (→ artist), `duration`
|
||||
(ms), `permalink_url`, `media.transcodings[]`, `policy`/`streamable`,
|
||||
`publisher_metadata` (optional album/release).
|
||||
- **transcoding**: `url` (a second API URL, not the CDN), `preset`,
|
||||
`format.{protocol, mime_type}`, `quality`. We select
|
||||
`protocol == "hls" && mime_type == "audio/mpeg"` (mp3-HLS), which SoundCloud
|
||||
offers for essentially every playable track.
|
||||
- **playlist**: `id`, `title`, `user.username`, `tracks[]` — often returned as
|
||||
**stubs** (`{id}` only); missing tracks are hydrated in batches of ≤50 via
|
||||
`GET /tracks?ids=<csv>&client_id=…`.
|
||||
- **resolve**: returns a track or a playlist object (discriminated by `kind`).
|
||||
|
||||
`client_id` acquisition (no login): `GET https://soundcloud.com`, find the
|
||||
referenced JS bundles, fetch them, regex `client_id:"(\w+)"`; `app_version`
|
||||
from `window.__sc_version="(\d+)"`. This is exactly the streamrip approach.
|
||||
|
||||
## Assumptions (decided here)
|
||||
|
||||
- The captures/creatable/editable/deletable TUI flows are provider-agnostic
|
||||
(confirmed by `/youtube`, `/fyyd`, `/abs`): search-term + resolve semantics
|
||||
cost no TUI or wire change. No proto change, no new `ProviderCommand`.
|
||||
- A missing/rotated/invalid `client_id` must never crash startup or a browse.
|
||||
The provider **self-heals** by scraping and by re-scraping on `401/403`; only
|
||||
if scraping itself fails does the `/soundcloud` subtree degrade (typed
|
||||
errors, skipped tracks), never the app.
|
||||
- **Login is optional.** With no `oauth_token`, personal nodes (`likes`,
|
||||
`playlists`) are simply **not shown**; public search/resolve/play still work.
|
||||
A token unlocks the personal nodes and is refreshed/persisted like tidal's.
|
||||
- HLS media/segment URLs and the `client_id`/`oauth_token` are **secrets or
|
||||
ephemeral signed URLs**: redact from `Debug`/config dumps, never log the built
|
||||
stream/segment URLs (hard rule: redact secrets from logs and error reports).
|
||||
- SoundCloud `id`s are numeric (URL-safe); only user-typed **search terms** and
|
||||
**pasted URLs** are percent-encoded into a path segment.
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1 — Crate `soundclouddy`, mounted at `/soundcloud`, non-fatal init
|
||||
|
||||
New workspace crate `soundclouddy` implementing `ProviderClient`, shaped on
|
||||
`fyyd`/`absdy` (remote, search-driven, plain leaf tracks). Wired into
|
||||
`ProviderOrchestrator` with a `sc_client: Option<Arc<soundclouddy::Client>>`
|
||||
field, `sc_owns()`/`sc_provider()` helpers, a `build()` block that reads
|
||||
`soundcloud.toml` (non-fatal), a `get_lib_root` child gated on
|
||||
`self.sc_client.is_some()`, and one routing arm in each dispatch method.
|
||||
`crabidy-server` settings gain `soundcloud` in `ALL_PROVIDERS` (now 8), in
|
||||
`ProviderToggles`, in `all()`, and in `provider_toggles()`. No `cli.rs`/
|
||||
`main.rs` change (providers are pure path-prefix subtrees).
|
||||
|
||||
### D2 — HTTP behind a trait, faked in tests; client_id lifecycle inside the seam
|
||||
|
||||
All network access goes through one seam — an `Sc` trait (`resolve`,
|
||||
`search_tracks`, `search_playlists`, `track_detail`, `playlist_detail`,
|
||||
`hydrate_tracks`, `resolve_stream_url`, and, when logged in, `my_likes`,
|
||||
`my_playlists`) behind `Box<dyn Sc>` — with a `reqwest`-based `ScApi` for
|
||||
production and a `FakeApi` in tests (as `absdy` hides `reqwest` behind `Abs`).
|
||||
The **client_id acquisition, caching, and re-scrape-on-401** live entirely
|
||||
inside `ScApi` so provider logic (tree shaping, path parsing, term store) is
|
||||
unit-tested with zero network. Errors map to `ProviderError::FetchError` at the
|
||||
boundary; malformed paths → `MalformedPath`; empty create/rename → `InvalidInput`.
|
||||
|
||||
### D3 — Tree shape (search, resolve, optional personal)
|
||||
|
||||
Track ids and playlist ids are both numeric, so leaf/container paths use a
|
||||
**type-tagged** canonical segment to disambiguate: `track/<id>` and
|
||||
`playlist/<id>`. Browse nodes point their children at these canonical paths.
|
||||
|
||||
- `/soundcloud` — children: `search` (creatable), `resolve` (creatable), and —
|
||||
**only if logged in** — `likes` and `playlists`. Not itself queueable.
|
||||
- `/soundcloud/search` / `/soundcloud/resolve` — `is_creatable`; children are
|
||||
the in-memory terms/URLs (`RwLock<Vec<String>>`, dedup), each editable and
|
||||
deletable, like tidal/youtube/fyyd/abs search terms.
|
||||
- `/soundcloud/search/<term>` — matching **tracks** as queueable leaves (and,
|
||||
optionally, matching playlists as containers).
|
||||
- `/soundcloud/resolve/<url>` — the resolved permalink: a single track leaf, or
|
||||
a playlist container.
|
||||
- `/soundcloud/likes` (login) — the user's liked **tracks**.
|
||||
- `/soundcloud/playlists` (login) — the user's playlists as containers.
|
||||
- `/soundcloud/playlist/<id>` — a playlist's tracks (queueable, downloadable);
|
||||
the canonical container path, reached from search/resolve/likes/playlists.
|
||||
- `/soundcloud/track/<id>` — the canonical **track leaf**. A track id alone is
|
||||
sufficient to resolve a stream, so every branch's track children point here
|
||||
and playback needs no browse context.
|
||||
|
||||
Terms/URLs are percent-encoded into one segment (`encode_segment`/
|
||||
`decode_segment`); numeric ids are already URL-safe.
|
||||
|
||||
### D4 — Playback via a new HLS source in `audio-player`
|
||||
|
||||
- `get_urls_for_track(/soundcloud/track/<id>)`: `GET /tracks/<id>` (or use a
|
||||
cached transcoding), pick the `hls + audio/mpeg` transcoding, `GET
|
||||
<transcoding.url>?client_id=…` → the `.m3u8` media URL, and **return that URL
|
||||
as `urls[0]`** (the player consumes only the first). One API round-trip —
|
||||
unlike abs's pure string-building — because the media URL is signed and
|
||||
ephemeral.
|
||||
- **New `audio-player` component `HlsStream`** — a `stream-download`
|
||||
`SourceStream`, sibling to `WindowedHttpStream`: on create it fetches the
|
||||
`.m3u8` (a media playlist), parses `#EXTINF`/segment URIs (resolving relative
|
||||
URIs against the playlist URL, and following one level if handed a master
|
||||
playlist); on poll it streams each mp3 segment's bytes in order, advancing at
|
||||
segment boundaries, finishing after the last. The concatenated bytes are a
|
||||
valid mp3 → rodio's symphonia mp3 decoder handles them unchanged.
|
||||
- **Routing**: `open_source` selects `HlsStream` when the URL path ends in
|
||||
`.m3u8` (SoundCloud's media URLs carry it); all other http URLs keep the
|
||||
windowed-HTTP path, and content-sniffing (opus vs the rest) is unchanged
|
||||
downstream. `#EXTM3U` content-sniff is a hardening fallback if needed.
|
||||
- **Duration** comes from the track metadata (`duration` ms → `Track.duration`),
|
||||
not from the stream, so the seek bar is correct even though the concatenated
|
||||
HLS stream carries no container duration.
|
||||
- Track fields: `title` = track title, `artist` = `user.username`, `album` from
|
||||
`publisher_metadata` when present else `None`, `duration` = ms→`Option<u32>`,
|
||||
`provider_item_id` = `"track:<id>"` (keys the capture store).
|
||||
|
||||
### D5 — Auth: client_id self-heal, optional OAuth login
|
||||
|
||||
- `Settings`: `client_id: Option<String>`, `app_version: Option<String>`
|
||||
(both **cached** after first scrape and round-tripped via `settings()` so we
|
||||
don't re-scrape every start), `oauth_token: Option<String>` (optional),
|
||||
and bounds (D6). Hand-written `Debug` redacts `client_id`/`oauth_token`.
|
||||
- **No login (baseline)**: if `client_id` is unset, `ScApi` scrapes it from
|
||||
`soundcloud.com` at init; on any `401/403` it re-scrapes **once** and retries
|
||||
(rotation recovery). The freshly scraped id is persisted.
|
||||
- **Optional login**: if `oauth_token` is present, `get_lib_root` adds `likes`
|
||||
and `playlists`, and personal calls send `Authorization: OAuth <token>`. If a
|
||||
refresh-token flow is configured later it mirrors tidal's persist-on-refresh;
|
||||
v1 accepts a static token and, on `401`, drops the personal subtree with a
|
||||
typed error (never a crash) — public browse is unaffected.
|
||||
|
||||
### D6 — Bounds and freshness
|
||||
|
||||
- `search_results` (default 50), `playlist_tracks_limit` (default 500, hydrated
|
||||
in ≤50-id batches), `call_timeout_secs` (default 30) bound each HTTP call, and
|
||||
`hls_total_deadline_secs` (default 300) bounds a whole HLS fetch (segments are
|
||||
retried with jitter under this deadline). Caps are `log`-ged so truncation is
|
||||
visible, not silent (hard rule: no silent caps).
|
||||
- Listings are fetched fresh per call (no cross-call cache), like the other
|
||||
remote providers; only search terms / resolve URLs are stored in memory. The
|
||||
chosen transcoding may be briefly cached per track id to save the extra
|
||||
round-trip on replay.
|
||||
|
||||
### D7 — Out of scope (explicitly)
|
||||
|
||||
- **opus-HLS** (`audio/ogg; codecs=opus`) and **progressive** transcodings: v1
|
||||
targets mp3-HLS uniformly (offered for ~all tracks). opus-HLS is a phase-2
|
||||
add that reuses the new `OpusSource` per segment; the `HlsStream` seam makes
|
||||
it additive.
|
||||
- **HLS seeking**: v1 HLS is forward-only and reports the source as
|
||||
non-seekable, so symphonia does not attempt an end-seek (which would need a
|
||||
known byte length). In-track seek is a later add (open at a segment offset).
|
||||
- Go+ / high-quality / lossless streams (need a premium account), uploads,
|
||||
comments, reposts feed, waveforms, social graph, and playback-progress sync.
|
||||
- Pagination past the configured caps (one page per listing).
|
||||
|
||||
## Structure
|
||||
|
||||
```d2
|
||||
direction: right
|
||||
|
||||
server: crabidy-server {
|
||||
orch: ProviderOrchestrator
|
||||
}
|
||||
|
||||
sc: "soundclouddy (crate)" {
|
||||
client: "Client\n(ProviderClient)"
|
||||
terms: "search terms + resolve URLs\n(in-memory)"
|
||||
api: "ScApi\n(reqwest seam: Sc trait,\nclient_id self-heal, opt OAuth)"
|
||||
client -> terms
|
||||
client -> api
|
||||
}
|
||||
|
||||
player: "audio-player" {
|
||||
hls: "HlsStream\n(new SourceStream)"
|
||||
dec: "rodio / symphonia\n(mp3, opus)"
|
||||
hls -> dec: "concatenated mp3 bytes"
|
||||
}
|
||||
|
||||
scapi: "SoundCloud\napi-v2 + HLS CDN" { shape: cloud }
|
||||
web: "soundcloud.com\n(HTML + JS)" { shape: cloud }
|
||||
|
||||
server.orch -> sc.client: "/soundcloud/..."
|
||||
sc.api -> scapi: "resolve / search / tracks / transcoding (JSON, timeout)"
|
||||
sc.api -> web: "scrape client_id (init, on 401)"
|
||||
server.orch -> player.hls: ".m3u8 media URL"
|
||||
player.hls -> scapi: "GET m3u8 + mp3 segments"
|
||||
```
|
||||
|
||||
## Key flow: search and play a track
|
||||
|
||||
```d2
|
||||
shape: sequence_diagram
|
||||
tui: TUI
|
||||
orch: Orchestrator
|
||||
s: soundclouddy
|
||||
api: "SoundCloud api-v2"
|
||||
hls: "HlsStream (audio-player)"
|
||||
cdn: "HLS CDN"
|
||||
|
||||
tui -> orch: "open /soundcloud/search"
|
||||
tui -> orch: "create term \"boards of canada\""
|
||||
orch -> s: "create_lib_node(search, term)"
|
||||
s -> tui: "term stored"
|
||||
tui -> orch: "open /soundcloud/search/<term>"
|
||||
orch -> s: "get_lib_node"
|
||||
s -> api: "GET /search/tracks?q=…&client_id="
|
||||
api -> s: "tracks (id, title, user, duration)"
|
||||
s -> tui: "tracks as queueable leaves (/soundcloud/track/<id>)"
|
||||
tui -> orch: "queue + play a track"
|
||||
orch -> s: "get_urls_for_track(/soundcloud/track/<id>)"
|
||||
s -> api: "GET /tracks/<id> → pick hls+audio/mpeg transcoding"
|
||||
s -> api: "GET <transcoding.url>?client_id= → { url: m3u8 }"
|
||||
s -> orch: "urls = [ m3u8 ]"
|
||||
orch -> hls: "player.play(m3u8)"
|
||||
hls -> cdn: "GET m3u8 (segments)"
|
||||
hls -> cdn: "GET segment 1..N (mp3, in order)"
|
||||
hls -> orch: "continuous mp3 → symphonia decodes"
|
||||
```
|
||||
|
||||
## Risks and open questions
|
||||
|
||||
- **client_id scraping fragility.** The scrape regexes depend on
|
||||
soundcloud.com's HTML/JS shape and can break on a redesign. Mitigation: a
|
||||
config override (`client_id` in `soundcloud.toml`) always wins, and failures
|
||||
are typed (subtree degrades, app survives). The scrape is the one piece with
|
||||
no test-double coverage of the *live* format — flagged as a **live-test gate**.
|
||||
- **Ephemeral media URLs.** The resolved `.m3u8` and its segments are signed and
|
||||
short-lived; playback must start promptly after resolution (like `/youtube`).
|
||||
A stale URL surfaces as a skipped track, never a crash. Never logged.
|
||||
- **HLS without a known length.** The concatenated stream has no total byte
|
||||
length; reported non-seekable so symphonia won't end-seek (the rodio-0.22
|
||||
panic the opus work documented). **Gate:** verify an end-to-end mp3-HLS play
|
||||
does not panic and reaches EOS cleanly.
|
||||
- **Master vs media playlist.** SoundCloud returns a media (segment) playlist
|
||||
for the chosen transcoding; `HlsStream` follows one level of master playlist
|
||||
defensively and picks the first variant.
|
||||
- **Playlist stubs.** Playlist detail may return track stubs; hydration in
|
||||
≤50-id batches is bounded by `playlist_tracks_limit`. A hydration miss drops
|
||||
that track (skipped), never an error.
|
||||
- **OAuth token lifetime.** v1 accepts a static token; expiry drops the personal
|
||||
subtree with a typed `401` (public browse unaffected). A device-flow/refresh
|
||||
upgrade mirrors tidal and is additive.
|
||||
- **Field / envelope drift.** DTOs decode defensively (`#[serde(default)]`);
|
||||
a renamed field is a local fix in `ScApi`. Live validation is a task-plan gate.
|
||||
|
|
@ -0,0 +1,318 @@
|
|||
//! An HLS [`SourceStream`]: fetches an `.m3u8` media playlist and streams its
|
||||
//! mp3 segments in order as one continuous byte stream.
|
||||
//!
|
||||
//! SoundCloud (and other HLS sources) serve audio as a playlist of short mp3
|
||||
//! segments rather than one file. mp3 frames byte-concatenate into a valid
|
||||
//! stream (the fact `ffmpeg -c copy` relies on), so streaming the segments in
|
||||
//! order yields bytes rodio's symphonia mp3 decoder handles unchanged
|
||||
//! (architecture/soundcloud-provider.md D4).
|
||||
//!
|
||||
//! Sibling to [`crate::windowed_http::WindowedHttpStream`]. This is **forward
|
||||
//! only**: it reports the source as non-seekable and length-less, so symphonia
|
||||
//! does not attempt an end-seek that would need a known total length (the
|
||||
//! rodio-0.22 panic the opus work documented).
|
||||
//!
|
||||
//! Never logs the playlist or segment URLs — they are signed and ephemeral.
|
||||
|
||||
use std::io;
|
||||
use std::pin::Pin;
|
||||
use std::task::{Context, Poll};
|
||||
|
||||
use bytes::Bytes;
|
||||
use futures::{Future, Stream};
|
||||
use stream_download::source::{DecodeError, SourceStream};
|
||||
use tracing::{debug, trace, warn};
|
||||
use url::Url;
|
||||
|
||||
/// Parameters for [`HlsStream::create`].
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct HlsParams {
|
||||
/// The `.m3u8` media-playlist URL.
|
||||
pub url: Url,
|
||||
pub client: reqwest::Client,
|
||||
}
|
||||
|
||||
impl HlsParams {
|
||||
pub fn new(url: Url, client: reqwest::Client) -> Self {
|
||||
Self { url, client }
|
||||
}
|
||||
}
|
||||
|
||||
/// Error creating the stream (playlist fetch/parse failed).
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
#[error("{0}")]
|
||||
pub struct HlsError(String);
|
||||
|
||||
impl DecodeError for HlsError {}
|
||||
|
||||
type BytesStream = Pin<Box<dyn Stream<Item = Result<Bytes, reqwest::Error>> + Send + Sync>>;
|
||||
type SegmentFuture = Pin<Box<dyn Future<Output = io::Result<BytesStream>> + Send + Sync>>;
|
||||
|
||||
/// A parsed playlist: either the media (segment) playlist we want, or a master
|
||||
/// playlist pointing at variant playlists (we follow the first).
|
||||
enum Playlist {
|
||||
Media(Vec<Url>),
|
||||
Master(Url),
|
||||
}
|
||||
|
||||
enum State {
|
||||
/// Ready to fetch `segments[cursor]`.
|
||||
Idle,
|
||||
/// Waiting for a segment response.
|
||||
Requesting(SegmentFuture),
|
||||
/// Draining a segment body.
|
||||
Streaming(BytesStream),
|
||||
Finished,
|
||||
}
|
||||
|
||||
/// See the module docs. Streams `segments[cursor]` bytes, advancing at segment
|
||||
/// boundaries; finishes after the last.
|
||||
pub struct HlsStream {
|
||||
client: reqwest::Client,
|
||||
/// Ordered mp3 segment URLs parsed from the media playlist.
|
||||
segments: Vec<Url>,
|
||||
/// Index of the next segment to fetch.
|
||||
cursor: usize,
|
||||
state: State,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for HlsStream {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("HlsStream")
|
||||
.field("segments", &self.segments.len())
|
||||
.field("cursor", &self.cursor)
|
||||
.finish_non_exhaustive()
|
||||
}
|
||||
}
|
||||
|
||||
impl HlsStream {
|
||||
/// Schedules the fetch of the segment at `cursor`, or finishes when none
|
||||
/// remain.
|
||||
fn schedule_next(&mut self) {
|
||||
let Some(url) = self.segments.get(self.cursor).cloned() else {
|
||||
self.state = State::Finished;
|
||||
return;
|
||||
};
|
||||
self.cursor += 1;
|
||||
self.state = State::Requesting(Box::pin(fetch_segment(self.client.clone(), url)));
|
||||
}
|
||||
}
|
||||
|
||||
/// GETs one segment, returning its body byte-stream. A non-success status is an
|
||||
/// error carrying the status only (never the signed URL).
|
||||
async fn fetch_segment(client: reqwest::Client, url: Url) -> io::Result<BytesStream> {
|
||||
trace!("fetching hls segment");
|
||||
let resp = client.get(url).send().await.map_err(|err| {
|
||||
io::Error::other(format!("segment request failed: {}", err.without_url()))
|
||||
})?;
|
||||
if !resp.status().is_success() {
|
||||
return Err(io::Error::other(format!(
|
||||
"segment request rejected: {}",
|
||||
resp.status()
|
||||
)));
|
||||
}
|
||||
Ok(Box::pin(resp.bytes_stream()))
|
||||
}
|
||||
|
||||
impl Stream for HlsStream {
|
||||
type Item = io::Result<Bytes>;
|
||||
|
||||
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
|
||||
let this = &mut *self;
|
||||
loop {
|
||||
match &mut this.state {
|
||||
State::Finished => return Poll::Ready(None),
|
||||
State::Idle => this.schedule_next(),
|
||||
State::Requesting(future) => match future.as_mut().poll(cx) {
|
||||
Poll::Pending => return Poll::Pending,
|
||||
Poll::Ready(Ok(stream)) => this.state = State::Streaming(stream),
|
||||
Poll::Ready(Err(err)) => {
|
||||
this.state = State::Finished;
|
||||
return Poll::Ready(Some(Err(err)));
|
||||
}
|
||||
},
|
||||
State::Streaming(stream) => match stream.as_mut().poll_next(cx) {
|
||||
Poll::Pending => return Poll::Pending,
|
||||
Poll::Ready(Some(Ok(bytes))) => return Poll::Ready(Some(Ok(bytes))),
|
||||
Poll::Ready(Some(Err(err))) => {
|
||||
return Poll::Ready(Some(Err(io::Error::other(format!(
|
||||
"segment body failed: {}",
|
||||
err.without_url()
|
||||
)))));
|
||||
}
|
||||
// Segment drained: move to the next one.
|
||||
Poll::Ready(None) => this.schedule_next(),
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl SourceStream for HlsStream {
|
||||
type Params = HlsParams;
|
||||
type StreamCreationError = HlsError;
|
||||
|
||||
async fn create(params: Self::Params) -> Result<Self, Self::StreamCreationError> {
|
||||
let segments = load_segments(¶ms.client, params.url).await?;
|
||||
debug!(segments = segments.len(), "hls stream open");
|
||||
if segments.is_empty() {
|
||||
return Err(HlsError("hls playlist has no segments".to_string()));
|
||||
}
|
||||
Ok(Self {
|
||||
client: params.client,
|
||||
segments,
|
||||
cursor: 0,
|
||||
state: State::Idle,
|
||||
})
|
||||
}
|
||||
|
||||
/// Unknown up front (segment sizes are not in the playlist), so `None` —
|
||||
/// which, with `supports_seek() == false`, keeps symphonia off the
|
||||
/// end-seek path.
|
||||
fn content_length(&self) -> Option<u64> {
|
||||
None
|
||||
}
|
||||
|
||||
/// Forward-only. In-track seeking (open at a segment offset) is a later,
|
||||
/// additive change (architecture/soundcloud-provider.md D7).
|
||||
async fn seek_range(&mut self, _start: u64, _end: Option<u64>) -> io::Result<()> {
|
||||
Err(io::Error::new(
|
||||
io::ErrorKind::Unsupported,
|
||||
"HLS stream is forward-only",
|
||||
))
|
||||
}
|
||||
|
||||
/// Best-effort resume after a dropped connection: re-open the current
|
||||
/// segment. `stream-download`'s temp storage already holds everything up to
|
||||
/// `current_position`, and its `Read` side serves that; this only needs to
|
||||
/// resume producing fresh bytes, so re-fetching the in-flight segment is
|
||||
/// acceptable (a short overlap at worst).
|
||||
async fn reconnect(&mut self, _current_position: u64) -> io::Result<()> {
|
||||
let restart = self.cursor.saturating_sub(1);
|
||||
warn!(
|
||||
segment = restart,
|
||||
"reconnecting hls stream at current segment"
|
||||
);
|
||||
self.cursor = restart;
|
||||
self.state = State::Idle;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn supports_seek(&self) -> bool {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
/// Fetches the playlist at `url`, following one level of master playlist, and
|
||||
/// returns the ordered media-segment URLs.
|
||||
async fn load_segments(client: &reqwest::Client, url: Url) -> Result<Vec<Url>, HlsError> {
|
||||
let body = fetch_text(client, url.clone()).await?;
|
||||
match parse_playlist(&body, &url)? {
|
||||
Playlist::Media(segments) => Ok(segments),
|
||||
Playlist::Master(variant) => {
|
||||
let body = fetch_text(client, variant.clone()).await?;
|
||||
match parse_playlist(&body, &variant)? {
|
||||
Playlist::Media(segments) => Ok(segments),
|
||||
// A master pointing at another master is not something
|
||||
// SoundCloud produces; refuse rather than recurse.
|
||||
Playlist::Master(_) => Err(HlsError("nested master playlist".to_string())),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn fetch_text(client: &reqwest::Client, url: Url) -> Result<String, HlsError> {
|
||||
let resp = client
|
||||
.get(url)
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| HlsError(format!("playlist request failed: {}", e.without_url())))?;
|
||||
if !resp.status().is_success() {
|
||||
return Err(HlsError(format!("playlist rejected: {}", resp.status())));
|
||||
}
|
||||
resp.text()
|
||||
.await
|
||||
.map_err(|e| HlsError(format!("playlist read failed: {e}")))
|
||||
}
|
||||
|
||||
/// Parses an m3u8 body. A master playlist (`#EXT-X-STREAM-INF`) yields the
|
||||
/// first variant URI; otherwise every non-comment line is a media segment.
|
||||
/// Relative URIs resolve against `base`.
|
||||
fn parse_playlist(body: &str, base: &Url) -> Result<Playlist, HlsError> {
|
||||
let is_master = body.contains("#EXT-X-STREAM-INF");
|
||||
let mut uris = Vec::new();
|
||||
for line in body.lines() {
|
||||
let line = line.trim();
|
||||
if line.is_empty() || line.starts_with('#') {
|
||||
continue;
|
||||
}
|
||||
let resolved = base
|
||||
.join(line)
|
||||
.map_err(|e| HlsError(format!("bad segment URI: {e}")))?;
|
||||
uris.push(resolved);
|
||||
if is_master {
|
||||
// Only the first variant is needed.
|
||||
break;
|
||||
}
|
||||
}
|
||||
if is_master {
|
||||
uris.into_iter()
|
||||
.next()
|
||||
.map(Playlist::Master)
|
||||
.ok_or_else(|| HlsError("master playlist has no variant".to_string()))
|
||||
} else {
|
||||
Ok(Playlist::Media(uris))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn base() -> Url {
|
||||
Url::parse("https://cf-hls.sndcdn.com/media/0/playlist.m3u8?token=x").unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_media_playlist_absolute_and_relative() {
|
||||
let body = "#EXTM3U\n\
|
||||
#EXT-X-VERSION:6\n\
|
||||
#EXTINF:10.0,\n\
|
||||
https://cdn.sndcdn.com/media/0/1.ts\n\
|
||||
#EXTINF:9.9,\n\
|
||||
2.ts\n\
|
||||
#EXT-X-ENDLIST\n";
|
||||
let Playlist::Media(segs) = parse_playlist(body, &base()).unwrap() else {
|
||||
panic!("expected media playlist");
|
||||
};
|
||||
assert_eq!(segs.len(), 2);
|
||||
assert_eq!(segs[0].as_str(), "https://cdn.sndcdn.com/media/0/1.ts");
|
||||
// Relative URI resolved against the playlist URL.
|
||||
assert_eq!(segs[1].as_str(), "https://cf-hls.sndcdn.com/media/0/2.ts");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn follows_first_master_variant() {
|
||||
let body = "#EXTM3U\n\
|
||||
#EXT-X-STREAM-INF:BANDWIDTH=128000\n\
|
||||
variant-128.m3u8\n\
|
||||
#EXT-X-STREAM-INF:BANDWIDTH=64000\n\
|
||||
variant-64.m3u8\n";
|
||||
let Playlist::Master(url) = parse_playlist(body, &base()).unwrap() else {
|
||||
panic!("expected master playlist");
|
||||
};
|
||||
assert_eq!(
|
||||
url.as_str(),
|
||||
"https://cf-hls.sndcdn.com/media/0/variant-128.m3u8"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ignores_comments_and_blank_lines() {
|
||||
let body = "#EXTM3U\n\n#EXTINF:1,\nonly.ts\n\n";
|
||||
let Playlist::Media(segs) = parse_playlist(body, &base()).unwrap() else {
|
||||
panic!("expected media playlist");
|
||||
};
|
||||
assert_eq!(segs.len(), 1);
|
||||
}
|
||||
}
|
||||
|
|
@ -1,3 +1,4 @@
|
|||
mod hls;
|
||||
mod opus_source;
|
||||
mod player;
|
||||
mod player_engine;
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ use rodio::{Decoder, Source};
|
|||
use stream_download::storage::temp::TempStorageProvider;
|
||||
use stream_download::{Settings, StreamDownload};
|
||||
|
||||
use crate::hls::{HlsParams, HlsStream};
|
||||
use crate::opus_source::{is_ogg_opus, OpusSource};
|
||||
use crate::spectrum_tap::{SpectrumTap, TappingSource};
|
||||
use crate::windowed_http::{WindowedHttpParams, WindowedHttpStream};
|
||||
|
|
@ -309,32 +310,57 @@ impl PlayerEngine {
|
|||
host = url.host_str().unwrap_or("?"),
|
||||
"opening network stream"
|
||||
);
|
||||
// Windowed fetching: some CDNs (googlevideo) reject plain
|
||||
// and open-ended requests with 403 and only serve bounded
|
||||
// ranges (see audio-player/src/windowed_http.rs).
|
||||
let params = WindowedHttpParams::new(url.clone(), self.http.clone());
|
||||
// HLS (`.m3u8`, e.g. SoundCloud) is a playlist of mp3 segments,
|
||||
// streamed in order by HlsStream; every other http URL uses the
|
||||
// windowed fetcher (some CDNs like googlevideo 403 plain and
|
||||
// open-ended requests, serving only bounded ranges — see
|
||||
// audio-player/src/windowed_http.rs).
|
||||
let is_hls = Path::new(url.path())
|
||||
.extension()
|
||||
.is_some_and(|ext| ext.eq_ignore_ascii_case("m3u8"));
|
||||
let reader = self.runtime.block_on(async {
|
||||
tokio::time::timeout(
|
||||
STREAM_OPEN_TIMEOUT,
|
||||
StreamDownload::new::<WindowedHttpStream>(
|
||||
params,
|
||||
// Each arm normalizes its distinct `StreamInitError<S>` to
|
||||
// `anyhow` so the branches unify.
|
||||
let open = async {
|
||||
if is_hls {
|
||||
StreamDownload::new::<HlsStream>(
|
||||
HlsParams::new(url.clone(), self.http.clone()),
|
||||
TempStorageProvider::new(),
|
||||
Settings::default(),
|
||||
),
|
||||
)
|
||||
.await
|
||||
.map_err(|_| anyhow!("timed out opening stream after {STREAM_OPEN_TIMEOUT:?}"))?
|
||||
.map_err(|err| anyhow!("{err}"))
|
||||
} else {
|
||||
StreamDownload::new::<WindowedHttpStream>(
|
||||
WindowedHttpParams::new(url.clone(), self.http.clone()),
|
||||
TempStorageProvider::new(),
|
||||
Settings::default(),
|
||||
)
|
||||
.await
|
||||
.map_err(|err| anyhow!("{err}"))
|
||||
}
|
||||
};
|
||||
tokio::time::timeout(STREAM_OPEN_TIMEOUT, open)
|
||||
.await
|
||||
.map_err(|_| {
|
||||
anyhow!("timed out opening stream after {STREAM_OPEN_TIMEOUT:?}")
|
||||
})?
|
||||
.context("failed to open http stream")
|
||||
})?;
|
||||
let byte_len = reader.content_length();
|
||||
let hint = Path::new(url.path())
|
||||
.extension()
|
||||
.and_then(|e| e.to_str())
|
||||
.map(str::to_owned);
|
||||
// HLS carries mp3 segments and no total length; hint mp3 and
|
||||
// mark non-seekable so symphonia never end-seeks (which panics
|
||||
// rodio 0.22 on a length-less stream).
|
||||
let hint = if is_hls {
|
||||
Some("mp3")
|
||||
} else {
|
||||
Path::new(url.path()).extension().and_then(|e| e.to_str())
|
||||
};
|
||||
self.build_source(
|
||||
reader,
|
||||
byte_len,
|
||||
hint.as_deref(),
|
||||
hint,
|
||||
!is_hls,
|
||||
"failed to decode http stream",
|
||||
)
|
||||
}
|
||||
|
|
@ -352,6 +378,7 @@ impl PlayerEngine {
|
|||
BufReader::new(file),
|
||||
byte_len,
|
||||
hint.as_deref(),
|
||||
true,
|
||||
"failed to decode file",
|
||||
)
|
||||
}
|
||||
|
|
@ -369,6 +396,7 @@ impl PlayerEngine {
|
|||
mut reader: R,
|
||||
byte_len: Option<u64>,
|
||||
hint: Option<&str>,
|
||||
seekable: bool,
|
||||
decode_err: &'static str,
|
||||
) -> Result<(Box<dyn Source + Send>, Option<Duration>)>
|
||||
where
|
||||
|
|
@ -382,7 +410,7 @@ impl PlayerEngine {
|
|||
|
||||
if is_ogg_opus(&header[..n]) {
|
||||
debug!("decoding Ogg-Opus via libopus");
|
||||
let source = OpusSource::new(reader, byte_len, true)?;
|
||||
let source = OpusSource::new(reader, byte_len, seekable)?;
|
||||
let duration = source.total_duration();
|
||||
let tapped: Box<dyn Source + Send> =
|
||||
Box::new(TappingSource::new(source, self.spectrum.clone()));
|
||||
|
|
@ -392,8 +420,9 @@ impl PlayerEngine {
|
|||
// Symphonia probes the container length during init; without a known
|
||||
// byte length it seeks from the end, which rodio 0.22 turns into an
|
||||
// `unreachable!` panic on a streamed source. Handing it the content
|
||||
// length up front avoids that seek entirely.
|
||||
let mut builder = Decoder::builder().with_data(reader).with_seekable(true);
|
||||
// length up front avoids that seek entirely; for length-less streams
|
||||
// (HLS) `seekable` is false so symphonia never attempts that seek.
|
||||
let mut builder = Decoder::builder().with_data(reader).with_seekable(seekable);
|
||||
if let Some(len) = byte_len {
|
||||
builder = builder.with_byte_len(len);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -40,6 +40,7 @@ futures.workspace = true
|
|||
rand.workspace = true
|
||||
reqwest.workspace = true
|
||||
serde.workspace = true
|
||||
soundclouddy.workspace = true
|
||||
thiserror.workspace = true
|
||||
tidaldy.workspace = true
|
||||
tokio = { workspace = true, features = ["full"] }
|
||||
|
|
|
|||
|
|
@ -44,6 +44,10 @@ pub struct ProviderOrchestrator {
|
|||
/// missing/incomplete — non-fatal, it only costs the `/abs` subtree
|
||||
/// (architecture/audiobookshelf-provider.md D1).
|
||||
abs_client: Option<Arc<absdy::Client>>,
|
||||
/// The SoundCloud provider; `None` when disabled or no `client_id` could be
|
||||
/// obtained — non-fatal, it only costs the `/soundcloud` subtree
|
||||
/// (architecture/soundcloud-provider.md D1).
|
||||
sc_client: Option<Arc<soundclouddy::Client>>,
|
||||
}
|
||||
|
||||
/// Whether a path belongs to the filesystem provider.
|
||||
|
|
@ -77,6 +81,11 @@ fn abs_owns(path: &str) -> bool {
|
|||
path == absdy::PROVIDER_ROOT || path.starts_with("/abs/")
|
||||
}
|
||||
|
||||
/// Whether a path belongs to the SoundCloud provider.
|
||||
fn sc_owns(path: &str) -> bool {
|
||||
path == soundclouddy::PROVIDER_ROOT || path.starts_with("/soundcloud/")
|
||||
}
|
||||
|
||||
impl ProviderOrchestrator {
|
||||
/// The tidal client, or `MalformedPath` (with a warning) when the
|
||||
/// provider is disabled — a `/tidal` path then has no owner.
|
||||
|
|
@ -145,6 +154,15 @@ impl ProviderOrchestrator {
|
|||
ProviderError::MalformedPath
|
||||
})
|
||||
}
|
||||
|
||||
/// The SoundCloud client, or `MalformedPath` (with a warning) when the
|
||||
/// provider is disabled — a `/soundcloud` path then has no owner.
|
||||
fn sc_provider(&self) -> Result<&soundclouddy::Client, ProviderError> {
|
||||
self.sc_client.as_deref().ok_or_else(|| {
|
||||
warn!("soundcloud provider is disabled");
|
||||
ProviderError::MalformedPath
|
||||
})
|
||||
}
|
||||
pub fn run(self) {
|
||||
tokio::spawn(async move {
|
||||
// Behind an Arc so long-running resolves can be spawned onto
|
||||
|
|
@ -459,6 +477,29 @@ impl ProviderOrchestrator {
|
|||
} else {
|
||||
None
|
||||
};
|
||||
// SoundCloud: non-fatal. Works with no config (scrapes a `client_id`);
|
||||
// a total failure to obtain one disables the `/soundcloud` subtree
|
||||
// only (architecture/soundcloud-provider.md D1). `init` may write back
|
||||
// a freshly scraped `client_id` so it persists.
|
||||
let sc_client = if enabled.soundcloud {
|
||||
let sc_config_file = config_dir.join("soundcloud.toml");
|
||||
debug!(config_file = %sc_config_file.display(), "loading soundcloud config");
|
||||
let raw_sc_settings = fs::read_to_string(&sc_config_file).unwrap_or_default();
|
||||
match soundclouddy::Client::init(&raw_sc_settings).await {
|
||||
Ok(client) => {
|
||||
if let Err(err) = tokio::fs::write(&sc_config_file, client.settings()).await {
|
||||
error!("failed to write soundcloud config file: {err}");
|
||||
}
|
||||
Some(Arc::new(client))
|
||||
}
|
||||
Err(err) => {
|
||||
warn!("soundcloud provider disabled: {err}");
|
||||
None
|
||||
}
|
||||
}
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let (provider_tx, provider_rx) = flume::bounded(100);
|
||||
Ok(Self {
|
||||
provider_rx,
|
||||
|
|
@ -471,6 +512,7 @@ impl ProviderOrchestrator {
|
|||
youtube_client,
|
||||
fyyd_client,
|
||||
abs_client,
|
||||
sc_client,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
|
@ -531,6 +573,12 @@ impl ProviderClient for ProviderOrchestrator {
|
|||
.as_ref()
|
||||
.is_some_and(|abs| abs.is_track_path(path));
|
||||
}
|
||||
if sc_owns(path) {
|
||||
return self
|
||||
.sc_client
|
||||
.as_ref()
|
||||
.is_some_and(|sc| sc.is_track_path(path));
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
|
|
@ -566,6 +614,9 @@ impl ProviderClient for ProviderOrchestrator {
|
|||
if abs_owns(track_path) {
|
||||
return self.abs_provider()?.get_urls_for_track(track_path).await;
|
||||
}
|
||||
if sc_owns(track_path) {
|
||||
return self.sc_provider()?.get_urls_for_track(track_path).await;
|
||||
}
|
||||
warn!(path = track_path, "no provider owns this track path");
|
||||
Err(ProviderError::MalformedPath)
|
||||
}
|
||||
|
|
@ -611,6 +662,9 @@ impl ProviderClient for ProviderOrchestrator {
|
|||
.get_metadata_for_track(track_path)
|
||||
.await;
|
||||
}
|
||||
if sc_owns(track_path) {
|
||||
return self.sc_provider()?.get_metadata_for_track(track_path).await;
|
||||
}
|
||||
warn!(path = track_path, "no provider owns this track path");
|
||||
Err(ProviderError::MalformedPath)
|
||||
}
|
||||
|
|
@ -658,6 +712,14 @@ impl ProviderClient for ProviderOrchestrator {
|
|||
LibraryNodeChild::new(absdy::PROVIDER_ROOT.to_owned(), "abs".to_owned(), false);
|
||||
root_node.children.push(child);
|
||||
}
|
||||
if self.sc_client.is_some() {
|
||||
let child = LibraryNodeChild::new(
|
||||
soundclouddy::PROVIDER_ROOT.to_owned(),
|
||||
"soundcloud".to_owned(),
|
||||
false,
|
||||
);
|
||||
root_node.children.push(child);
|
||||
}
|
||||
root_node
|
||||
}
|
||||
|
||||
|
|
@ -681,6 +743,8 @@ impl ProviderClient for ProviderOrchestrator {
|
|||
self.fyyd_provider()?.get_lib_node(path).await?
|
||||
} else if abs_owns(path) {
|
||||
self.abs_provider()?.get_lib_node(path).await?
|
||||
} else if sc_owns(path) {
|
||||
self.sc_provider()?.get_lib_node(path).await?
|
||||
} else {
|
||||
warn!(path, "no provider owns this path");
|
||||
return Err(ProviderError::MalformedPath);
|
||||
|
|
@ -743,6 +807,12 @@ impl ProviderClient for ProviderOrchestrator {
|
|||
.create_lib_node(parent_path, title)
|
||||
.await;
|
||||
}
|
||||
if sc_owns(parent_path) {
|
||||
return self
|
||||
.sc_provider()?
|
||||
.create_lib_node(parent_path, title)
|
||||
.await;
|
||||
}
|
||||
warn!(parent_path, "no provider supports creating nodes here");
|
||||
Err(ProviderError::NotSupported)
|
||||
}
|
||||
|
|
@ -788,6 +858,9 @@ impl ProviderClient for ProviderOrchestrator {
|
|||
if abs_owns(path) {
|
||||
return self.abs_provider()?.rename_lib_node(path, new_title).await;
|
||||
}
|
||||
if sc_owns(path) {
|
||||
return self.sc_provider()?.rename_lib_node(path, new_title).await;
|
||||
}
|
||||
warn!(path, "no provider supports renaming this node");
|
||||
Err(ProviderError::NotSupported)
|
||||
}
|
||||
|
|
@ -842,6 +915,12 @@ impl ProviderClient for ProviderOrchestrator {
|
|||
.resolve_tracks_into(path, chunk_tx)
|
||||
.await;
|
||||
}
|
||||
if sc_owns(path) {
|
||||
return self
|
||||
.sc_provider()?
|
||||
.resolve_tracks_into(path, chunk_tx)
|
||||
.await;
|
||||
}
|
||||
warn!(path, "no provider owns this path");
|
||||
Err(ProviderError::MalformedPath)
|
||||
}
|
||||
|
|
@ -871,6 +950,9 @@ impl ProviderClient for ProviderOrchestrator {
|
|||
if abs_owns(path) {
|
||||
return self.abs_provider()?.delete_lib_node(path).await;
|
||||
}
|
||||
if sc_owns(path) {
|
||||
return self.sc_provider()?.delete_lib_node(path).await;
|
||||
}
|
||||
warn!(path, "no provider supports deleting this node");
|
||||
Err(ProviderError::NotSupported)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -16,11 +16,18 @@ use serde::{Deserialize, Serialize};
|
|||
pub const SETTINGS_FILE: &str = "crabidy-server.toml";
|
||||
|
||||
/// Every built-in provider, in the order the default config lists them. Each
|
||||
/// name is a library root (`/tidal`, `/youtube`, `/fyyd`, `/abs`, `/fs`,
|
||||
/// `/crabidy`, `/orphans`). `orphans` is a view over the store, so it needs
|
||||
/// `crabidy`.
|
||||
pub const ALL_PROVIDERS: [&str; 7] = [
|
||||
"tidal", "youtube", "fyyd", "abs", "fs", "crabidy", "orphans",
|
||||
/// name is a library root (`/tidal`, `/youtube`, `/fyyd`, `/abs`,
|
||||
/// `/soundcloud`, `/fs`, `/crabidy`, `/orphans`). `orphans` is a view over the
|
||||
/// store, so it needs `crabidy`.
|
||||
pub const ALL_PROVIDERS: [&str; 8] = [
|
||||
"tidal",
|
||||
"youtube",
|
||||
"fyyd",
|
||||
"abs",
|
||||
"soundcloud",
|
||||
"fs",
|
||||
"crabidy",
|
||||
"orphans",
|
||||
];
|
||||
|
||||
/// Contents of `crabidy-server.toml`.
|
||||
|
|
@ -70,6 +77,7 @@ pub struct ProviderToggles {
|
|||
pub youtube: bool,
|
||||
pub fyyd: bool,
|
||||
pub abs: bool,
|
||||
pub soundcloud: bool,
|
||||
pub fs: bool,
|
||||
pub crabidy: bool,
|
||||
pub orphans: bool,
|
||||
|
|
@ -84,6 +92,7 @@ impl ProviderToggles {
|
|||
youtube: true,
|
||||
fyyd: true,
|
||||
abs: true,
|
||||
soundcloud: true,
|
||||
fs: true,
|
||||
crabidy: true,
|
||||
orphans: true,
|
||||
|
|
@ -191,6 +200,7 @@ impl ServerSettings {
|
|||
youtube: self.provider_enabled("youtube"),
|
||||
fyyd: self.provider_enabled("fyyd"),
|
||||
abs: self.provider_enabled("abs"),
|
||||
soundcloud: self.provider_enabled("soundcloud"),
|
||||
fs: self.provider_enabled("fs"),
|
||||
crabidy: self.provider_enabled("crabidy"),
|
||||
orphans: self.provider_enabled("orphans"),
|
||||
|
|
|
|||
|
|
@ -0,0 +1,111 @@
|
|||
# Task plan — soundcloud provider
|
||||
|
||||
**Status (2026-07-24): all phases A–E implemented.** Everything offline is
|
||||
verified (unit tests, parsers, server build, fmt/clippy/machete). The live-only
|
||||
gates — the real `client_id` scrape (B2/B3), live JSON shapes (B4/B5), and
|
||||
mp3-HLS play-to-EOS (C4), plus browsing `/soundcloud` (D4) and `tests/live.rs`
|
||||
(E1) — need real SoundCloud access and are left to run on a machine with creds
|
||||
and audio. See `plan/summary.md`.
|
||||
|
||||
Executes architecture/soundcloud-provider.md against the stubs in
|
||||
`soundclouddy/` and `audio-player/src/hls.rs`, satisfying the gates in
|
||||
`quality/soundcloud-provider.md`. Ordered by dependency. Verification column
|
||||
names the test(s) / gate(s) each task must satisfy.
|
||||
|
||||
## Phase A — Provider logic (no network; unit-testable now)
|
||||
|
||||
- [ ] **A1 — `Settings` + accessors.** Bounds accessors (`search_results`,
|
||||
`playlist_tracks`, `call_timeout`, `hls_deadline`) with the `DEFAULT_*`
|
||||
consts; redacting `Debug` already present. *(G1; `settings_debug_redacts_secrets`.)*
|
||||
- [ ] **A2 — `get_lib_root` builds root children directly** (sync, no network):
|
||||
`search` + `resolve`, plus `likes` + `playlists` iff `logged_in`. Serve the
|
||||
Root node from here (decision #1) and have `get_lib_node(Root)` delegate to it.
|
||||
*(G10; `root_without_login_has_no_personal_nodes`, `root_with_login_shows_personal_nodes`.)*
|
||||
- [ ] **A3 — `sc_track` mapper** ScTrack→proto `Track` at `/soundcloud/track/<id>`,
|
||||
`provider_item_id = "track:<id>"`, ms→`Option<u32>`. *(G12.)*
|
||||
- [ ] **A4 — `search_node` + term store** (create/rename/delete, dedup, implicit
|
||||
recreation). *(G17; `create_rename_delete_search_term`.)*
|
||||
- [ ] **A5 — `search_term_node`** lists tracks (into `node.tracks`, canonical
|
||||
paths, queueable node) **and** playlists (into `node.children`, canonical
|
||||
container paths, queueable). *(G13, G12, G16;
|
||||
`search_term_lists_tracks_and_playlists_with_canonical_paths`.)*
|
||||
- [ ] **A6 — `resolve_node` + URL store** (create/rename/delete), and
|
||||
**`resolve_entry_node`**: track→`node.tracks=[t]`; playlist→child container.
|
||||
*(G14, G17; `resolve_entry_shapes_track_and_playlist`, `create_resolve_url_entry`.)*
|
||||
- [ ] **A7 — `playlist_node`** fetch detail, hydrate stub ids in ≤50 batches up
|
||||
to `playlist_tracks`, `log` truncation. *(G15; `playlist_node_hydrates_stub_tracks`.)*
|
||||
- [ ] **A8 — `likes_node` / `playlists_node`** (login only). *(G10.)*
|
||||
- [ ] **A9 — `get_urls_for_track` / `get_metadata_for_track`** via
|
||||
`resolve_stream_url` / `track_detail`; non-track path → `MalformedPath`.
|
||||
*(G18; `get_urls_for_track_returns_the_m3u8`.)*
|
||||
- [ ] **A10 — Download blessing** in `get_lib_node` (node + children), and typed
|
||||
`fetch_err` mapping (no panics). *(G4, G16; `backend_failures_are_typed`.)*
|
||||
- [ ] **A11 — Gate:** `cargo test -p soundclouddy` — all 15 tests pass. *(G11, G22.)*
|
||||
|
||||
## Phase B — HTTP seam `ScApi` (needs live SoundCloud to fully verify)
|
||||
|
||||
- [ ] **B1 — `ScApi::new`** builds the `reqwest` client with per-call timeout +
|
||||
user-agent; holds `client_id`/`app_version` behind `RwLock`, optional OAuth.
|
||||
*(G3, G6.)*
|
||||
- [ ] **B2 — `scrape_client_id` free fn** — fetch soundcloud.com, find script
|
||||
bundles, regex `client_id` + `app_version`. Unit-test the **parser** against a
|
||||
captured HTML/JS fixture (no network). *(G7; new `scrape_parses_fixture` test.)*
|
||||
- [ ] **B3 — Signed GET helper**: append `client_id`/`app_version`, OAuth header
|
||||
when present; map status→`FetchError` (404/401/403); **re-scrape once on
|
||||
401/403** then retry. *(G8; `reauth_rescrapes_once_then_gives_up` over a
|
||||
mock/loopback.)*
|
||||
- [ ] **B4 — Defensive wire DTOs** (`#[serde(default)]`, drop id-less entries)
|
||||
and `into_*` converters for track/playlist/resolve/search/transcoding.
|
||||
*(G5.)*
|
||||
- [ ] **B5 — `resolve_stream_url`**: pick `hls`+`audio/mpeg` transcoding, GET its
|
||||
url, return the `.m3u8`; `NotStreamable` when absent. *(G18.)*
|
||||
- [ ] **B6 — `hydrate_tracks`** batched `/tracks?ids=` (≤50). *(G15.)*
|
||||
- [ ] **B7 — `settings()`** folds cached `client_id`/`app_version` back in.
|
||||
*(G9; `settings_round_trip_persists_scraped_id`.)*
|
||||
- [ ] **B8 — `init`** parses TOML, builds `ScApi`, ensures a `client_id`
|
||||
(scrape if none), non-fatal on total failure. *(G7.)*
|
||||
|
||||
## Phase C — HLS playback in `audio-player`
|
||||
|
||||
- [ ] **C1 — `parse_media_playlist`** hand-written line parser (`#EXTINF` + URI,
|
||||
resolve relative against base, one-level master fallback). Unit test with a
|
||||
fixture playlist string. *(G19; new `hls::tests::parses_media_playlist`.)*
|
||||
- [ ] **C2 — `HlsStream` state machine** (`Stream` + `SourceStream`): fetch
|
||||
segments in order, advance at boundaries, finish after last; `reconnect`
|
||||
re-opens the current segment; forward-only, `content_length None`,
|
||||
`supports_seek false`; total-deadline bound. *(G6, G19, G20.)*
|
||||
- [ ] **C3 — Route `.m3u8` in `open_source`** to `HlsStream` (via `StreamDownload`)
|
||||
before the windowed-HTTP path; opus/other sniffing downstream unchanged. Wire
|
||||
`mod hls;` (already declared). *(G21.)*
|
||||
- [ ] **C4 — Gate (integration):** an ffmpeg-built mp3-HLS fixture (or a live
|
||||
track) plays to EOS with no panic; add a `#[ignore]` play-to-EOS test.
|
||||
*(G19, G20.)*
|
||||
|
||||
## Phase D — Server wiring
|
||||
|
||||
- [ ] **D1 — Workspace dep** `soundclouddy = { path = "soundclouddy" }` in root
|
||||
`Cargo.toml` `[workspace.dependencies]`; add to `crabidy-server` deps.
|
||||
- [ ] **D2 — `settings.rs`**: `"soundcloud"` in `ALL_PROVIDERS` (→8),
|
||||
`ProviderToggles.sc`, `all()`, `provider_toggles()`.
|
||||
- [ ] **D3 — `provider.rs`**: `sc_client` field, `sc_owns`, `sc_provider`,
|
||||
`build()` block reading `soundcloud.toml` (non-fatal, round-trip
|
||||
`settings()`), `get_lib_root` child gated on `is_some()`, and a routing arm in
|
||||
each dispatch method. *(mirrors abs.)*
|
||||
- [ ] **D4 — Gate:** `cargo build -p crabidy-server`; browse `/soundcloud` in the
|
||||
running app (manual/`/run`).
|
||||
|
||||
## Phase E — Validation & docs
|
||||
|
||||
- [ ] **E1 — `tests/live.rs`** (`#[ignore]`): reads `SOUNDCLOUD_OAUTH`/optional
|
||||
`SOUNDCLOUD_CLIENT_ID` from env, exercises scrape→search→resolve→stream_url
|
||||
end-to-end; confirms DTOs match live JSON. *(G7 live gate.)*
|
||||
- [ ] **E2 — Quality sweep:** `cargo fmt`, `clippy`, `machete`; tick
|
||||
`quality/soundcloud-provider.md`. *(G23.)*
|
||||
- [ ] **E3 — `plan/summary.md`** records deviations from this plan.
|
||||
|
||||
## Notes
|
||||
|
||||
- **Live-verification boundary:** Phases A and C1 are fully verifiable offline.
|
||||
B3/B5/C4/D4/E1 need live SoundCloud access (no creds in CI/sandbox) — deliver
|
||||
as code + fixture/mock tests, run the live gate on a real machine.
|
||||
- Do **not** commit secrets; `soundcloud.toml` stays out of git.
|
||||
|
|
@ -1055,3 +1055,70 @@ smoke test on the running server with a configured `abs.toml`.
|
|||
Verification: `absdy` 14 tests + `crabidy-server` 12 (+ existing) +
|
||||
`crabidy-core` 77 green; clippy and rustfmt clean on `absdy` and
|
||||
`crabidy-server`.
|
||||
|
||||
## opus playback (2026-07-24)
|
||||
|
||||
Fixed "can't play opus files from the abs server". Root cause: rodio decodes
|
||||
via symphonia 0.5, which ships **no Opus decoder**, and the abs provider serves
|
||||
raw `.opus` files — so `Decoder::build()` failed on them (every other format
|
||||
worked). Client-side fix, so it covers opus from **any** provider:
|
||||
|
||||
- **`audio-player/src/opus_source.rs`** (new): `OpusSource`, a rodio `Source`
|
||||
that demuxes Ogg with symphonia's own Ogg reader and decodes Opus via
|
||||
**`symphonia-adapter-libopus`** (libopus, registered into an explicit codec
|
||||
registry — rodio's `Decoder` won't take a custom registry, so opus is driven
|
||||
directly). Mirrors rodio 0.22's `SymphoniaDecoder` loop (channels/rate/
|
||||
duration/seek). First packet decoded up front so the struct always holds a
|
||||
real spec (symphonia's `SampleBuffer::new` divides by channel count).
|
||||
- **`player_engine.rs`**: `open_source` refactored into a shared `build_source`
|
||||
that sniffs the first 64 bytes (`OggS`+`OpusHead`), rewinds, and routes opus
|
||||
to `OpusSource`, everything else to rodio. Shared across the http + file
|
||||
paths. Content-sniffing (not the extension hint) is essential — the abs
|
||||
stream URL has no extension (the ino is a number).
|
||||
- **`devenv.nix`**: `cmake`+`ninja` (bundled libopus builds with CMake).
|
||||
|
||||
**Verified end-to-end** with ffmpeg-encoded fixtures: mono opus → 48 kHz, 2.02 s
|
||||
duration, correct sample count; stereo opus → 2ch/48 kHz, and **seek to 1 s of a
|
||||
3 s file** left ~2.02 s. fmt/clippy clean, 11 audio-player tests pass,
|
||||
`crabidy-server` builds (public API unchanged).
|
||||
|
||||
## soundcloud provider — dev-flow S1–S4 + implement Phase A (2026-07-24)
|
||||
|
||||
Ran the dev-flow for a `/soundcloud` provider (streamrip referenced for the
|
||||
API). Decisions confirmed with the user: (1) root children served sync from
|
||||
`get_lib_root`; (2) hand-written m3u8 parser (no dep); (3) `scrape_client_id`
|
||||
free fn; (4) search lists tracks **and** playlists; playback via a new
|
||||
`HlsStream`; client_id config-or-scrape with re-scrape-on-401; **login
|
||||
optional** (OAuth token unlocks likes/playlists, else public-only).
|
||||
|
||||
- **S1 architecture** `architecture/soundcloud-provider.md` (D1–D7, 2 diagrams).
|
||||
- **S2 api-design** `soundclouddy` crate stubs (`Sc` seam, `Client`,
|
||||
`ScPath`+real `parse_path`) + `audio-player/src/hls.rs` (`HlsStream` skeleton).
|
||||
- **S3 quality-gates** `quality/soundcloud-provider.md` (G1–G23) + acceptance
|
||||
tests.
|
||||
- **S4 task-plan** `plan/soundcloud.md` (phases A–E, each mapped to a gate/test).
|
||||
- **S5 implement — Phases A–E built**:
|
||||
- **A** provider tree/path logic (`lib.rs`) over the `Sc` seam.
|
||||
- **B** `ScApi`: reqwest client with per-call timeout + browser UA, signed GET
|
||||
with **re-scrape-on-401** (once), pure scrape parsers
|
||||
(`extract_app_version`/`extract_script_urls`/`extract_client_id`, unit-tested
|
||||
against fixtures), defensive wire DTOs, `resolve_stream_url` (picks
|
||||
hls+audio/mpeg → media m3u8), batched `hydrate_tracks`, and `init` that
|
||||
scrapes if needed and persists the id.
|
||||
- **C** `audio-player/src/hls.rs` `HlsStream` (`SourceStream`): fetches the
|
||||
m3u8, follows one master level, streams mp3 segments in order; forward-only,
|
||||
length-less, non-seekable. `parse_playlist` unit-tested. `open_source` routes
|
||||
`.m3u8` → HlsStream (non-seekable so symphonia never end-seeks); `build_source`
|
||||
gained a `seekable` flag.
|
||||
- **D** server wiring: `soundclouddy` dep, `settings.rs` (`ALL_PROVIDERS`→8,
|
||||
`ProviderToggles.soundcloud`), `provider.rs` (`sc_client`, `sc_owns`,
|
||||
`sc_provider`, non-fatal `build` block, root child, all 8 dispatch arms).
|
||||
- **E** `soundclouddy/tests/live.rs` (`#[ignore]`, env-gated); fmt/clippy/machete
|
||||
clean; dropped unused `serde_json`.
|
||||
|
||||
**Verified offline:** soundclouddy 19 tests + 4 api-parser tests, audio-player 14
|
||||
(incl. 3 HLS-parser), crabidy-server 77+4 — all green; `cargo build -p
|
||||
crabidy-server` clean. **Needs live SoundCloud to confirm (no creds/audio in
|
||||
sandbox):** the actual `client_id` scrape, live API JSON shapes, and mp3-HLS
|
||||
play-to-EOS — all exercised by `tests/live.rs` and the `#[ignore]` gates
|
||||
(quality G7/G8/G14/G19). Run those on a real machine.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,114 @@
|
|||
# Quality gates — soundcloud provider
|
||||
|
||||
Criteria an implementation of `soundclouddy`
|
||||
(architecture/soundcloud-provider.md) must satisfy. Each is pass/fail by
|
||||
reading/reasoning; automated coverage lives in `soundclouddy/src/tests.rs`
|
||||
(unit, no network, over a `FakeSc`) and `soundclouddy/tests/live.rs` (ignored,
|
||||
real SoundCloud).
|
||||
|
||||
**Status (2026-07-24):** G1, G3–G6, G10–G13, G15–G18, G20–G23 are
|
||||
implemented and verified offline (unit tests + fmt/clippy/machete). G7/G8/G9
|
||||
(scrape + re-auth + persist) are implemented; their **live** behaviour and
|
||||
G14/G19 (real resolve/playlist + mp3-HLS play-to-EOS) are confirmed only by
|
||||
`tests/live.rs` and a manual play-through on a machine with creds/audio. G6's
|
||||
per-call timeouts are enforced; a whole-HLS-fetch deadline is deferred (the
|
||||
reqwest client's per-segment timeout bounds each fetch).
|
||||
|
||||
## Secrets (hard rule — highest priority)
|
||||
|
||||
- [ ] **G1 — `client_id`/`oauth_token` never in `Debug`.** `Settings`, `Client`,
|
||||
and `ScApi` `Debug` output redact both. *(test:
|
||||
`settings_debug_redacts_secrets`.)*
|
||||
- [ ] **G2 — The signed media URL never reaches a log or error.** The `.m3u8`
|
||||
URL from `Sc::resolve_stream_url` (and the segment URLs in `HlsStream`) are
|
||||
never passed to `debug!`/`warn!`/`error!`. Segment fetch errors carry status
|
||||
only (use `reqwest::Error::without_url()` like `windowed_http.rs`).
|
||||
- [ ] **G3 — `ScApi::Debug` redacts `client_id` and `oauth_token`** (manual
|
||||
impl, not derived).
|
||||
|
||||
## Errors and robustness (hard rule: no panics on input/network)
|
||||
|
||||
- [ ] **G4 — No panics on bad input or network failures.** Backend failures map
|
||||
to a typed `ProviderError` (`FetchError` for fetches, `MalformedPath` for bad
|
||||
paths, `InvalidInput` for empty titles/URLs). No `unwrap`/`expect`/`panic!` on
|
||||
request, path, or response data. *(tests: `backend_failures_are_typed`,
|
||||
`foreign_and_malformed_paths_reject`.)*
|
||||
- [ ] **G5 — Defensive decoding.** Wire DTOs use `#[serde(default)]`; missing
|
||||
fields degrade (empty string / `None` / dropped entry), never error the whole
|
||||
call. Tracks/playlists without an id are dropped, not panicked on.
|
||||
- [ ] **G6 — Timeouts on every external call.** `ScApi` sets a per-call
|
||||
`reqwest` timeout from `call_timeout_secs` (default 30); `HlsStream` bounds
|
||||
the whole fetch by `hls_deadline_secs` (default 300).
|
||||
|
||||
## Auth: client_id self-heal, optional login (D5)
|
||||
|
||||
- [ ] **G7 — Works with no config.** With an empty `soundcloud.toml`, init
|
||||
scrapes a `client_id` and the provider serves public browse/play; a total
|
||||
scrape failure disables the provider **non-fatally** (never crashes startup).
|
||||
- [ ] **G8 — Re-scrape on 401.** A `401/403` triggers exactly **one** re-scrape
|
||||
and retry; a second failure surfaces as `FetchError::Unauthorized` (no
|
||||
infinite loop). *(test: `reauth_rescrapes_once_then_gives_up`.)*
|
||||
- [ ] **G9 — Scraped `client_id` is persisted.** `settings()` folds the cached
|
||||
`client_id`/`app_version` back into the TOML so the next start does not
|
||||
re-scrape. *(test: `settings_round_trip_persists_scraped_id`.)*
|
||||
- [ ] **G10 — Login is optional and gates only personal nodes.** No
|
||||
`oauth_token` → `get_lib_root` children are `search` + `resolve` only; a token
|
||||
→ `likes` + `playlists` also appear. Public browse/play is identical either
|
||||
way. *(tests: `root_without_login_has_no_personal_nodes`,
|
||||
`root_with_login_shows_personal_nodes`.)*
|
||||
|
||||
## Tree and path contract
|
||||
|
||||
- [ ] **G11 — Path parsing is total and tag-safe.** Every `/soundcloud/...`
|
||||
shape maps to a variant or `MalformedPath`; the literals
|
||||
`search`/`resolve`/`likes`/`playlists`/`track`/`playlist` never collide with a
|
||||
numeric id. Empty segments (`/soundcloud//x`) are `MalformedPath`. *(tests:
|
||||
the `*_parse` / `*_reject` cases.)*
|
||||
- [ ] **G12 — Canonical leaves.** Every track child points at
|
||||
`/soundcloud/track/<id>` and every playlist child at
|
||||
`/soundcloud/playlist/<id>`, so a track id alone resolves a stream (no browse
|
||||
context needed). *(test: `search_children_use_canonical_paths`.)*
|
||||
- [ ] **G13 — Search lists tracks *and* playlists.** `/soundcloud/search/<term>`
|
||||
children include matching tracks (queueable leaves) and matching playlists
|
||||
(containers). *(test: `search_term_lists_tracks_and_playlists`.)*
|
||||
- [ ] **G14 — Resolve handles both kinds.** `/soundcloud/resolve/<url>` yields a
|
||||
single track leaf or a playlist container per the resolved `kind`. *(test:
|
||||
`resolve_entry_shapes_track_and_playlist`.)*
|
||||
- [ ] **G15 — Playlists hydrate within bounds.** A playlist node hydrates stub
|
||||
ids in ≤50-id batches up to `playlist_tracks` (default 500); a truncated
|
||||
playlist is `log`-ged, not silently cut. *(test:
|
||||
`playlist_node_hydrates_stub_tracks`.)*
|
||||
- [ ] **G16 — Download blessing.** Every node serving tracks and each queueable
|
||||
child raises `is_downloadable`, matching abs/fyyd. *(test:
|
||||
`queueable_nodes_are_downloadable`.)*
|
||||
- [ ] **G17 — Search terms / resolve URLs are creatable/editable/deletable.**
|
||||
`/soundcloud/search` and `/soundcloud/resolve` are `is_creatable`; their term
|
||||
children are `is_editable` + `is_deletable`, with implicit recreation on stale
|
||||
paths. *(tests: `create_rename_delete_search_term`,
|
||||
`create_resolve_url_entry`.)*
|
||||
|
||||
## Playback: HLS (D4)
|
||||
|
||||
- [ ] **G18 — `get_urls_for_track` returns one m3u8.** For a `track/<id>` path it
|
||||
returns exactly `[m3u8_url]` via `resolve_stream_url`; a non-track path is
|
||||
`MalformedPath`; a non-streamable track surfaces a typed error (skipped, never
|
||||
a crash). *(test: `get_urls_for_track_returns_m3u8`.)*
|
||||
- [ ] **G19 — `HlsStream` streams segments in order as continuous mp3.** The
|
||||
media playlist is parsed (hand-written; relative URIs resolved against the
|
||||
playlist URL; one level of master-playlist fallback), segments are fetched in
|
||||
order, and the concatenated bytes decode as mp3. *(gate: live/integration —
|
||||
a real or fixture m3u8 plays to EOS without panic.)*
|
||||
- [ ] **G20 — `HlsStream` is forward-only and non-seekable.** `supports_seek()`
|
||||
is `false` and `content_length()` is `None`, so symphonia never attempts the
|
||||
end-seek that panics rodio 0.22 on a length-less stream. *(gate: verified in
|
||||
the play-to-EOS integration check.)*
|
||||
- [ ] **G21 — HLS routing.** `audio-player`'s `open_source` routes a `.m3u8`
|
||||
URL to `HlsStream` and every other http URL to the windowed-HTTP path;
|
||||
opus/other content-sniffing downstream is unchanged.
|
||||
|
||||
## Docs and hygiene
|
||||
|
||||
- [ ] **G22 — Public items documented.** Every `pub` item has a doc comment
|
||||
stating intent and error/edge behavior.
|
||||
- [ ] **G23 — `cargo fmt`/`clippy` clean** for `soundclouddy` and `audio-player`
|
||||
(workspace lint level), and `cargo machete` reports no unused deps.
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
[package]
|
||||
name = "soundclouddy"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
|
||||
[dependencies]
|
||||
async-trait.workspace = true
|
||||
crabidy-core = { path = "../crabidy-core" }
|
||||
reqwest.workspace = true
|
||||
serde.workspace = true
|
||||
thiserror.workspace = true
|
||||
tokio = { workspace = true, features = ["time", "sync"] }
|
||||
toml.workspace = true
|
||||
tracing.workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
tokio = { workspace = true, features = ["full"] }
|
||||
|
|
@ -0,0 +1,750 @@
|
|||
//! The SoundCloud HTTP seam.
|
||||
//!
|
||||
//! All network access goes through the [`Sc`] trait so the provider's
|
||||
//! tree/path logic is unit-tested with a fake and no network
|
||||
//! (architecture/soundcloud-provider.md D2). [`ScApi`] is the production
|
||||
//! `reqwest` implementation; tests supply their own [`Sc`].
|
||||
//!
|
||||
//! Two secrets live here and must never leak: the `client_id` (scraped from
|
||||
//! soundcloud.com and rotated) and the optional `oauth_token`. The resolved
|
||||
//! HLS media URL is signed and ephemeral — also never logged. `ScApi` owns
|
||||
//! `client_id` acquisition and the **re-scrape-on-401** recovery (D5), so the
|
||||
//! provider layer never sees a raw credential. The scrape *parsers* are pure
|
||||
//! functions (unit-tested against fixtures); only the fetch glue needs the
|
||||
//! network.
|
||||
|
||||
use std::fmt::{self, Debug};
|
||||
use std::time::Duration;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use serde::de::DeserializeOwned;
|
||||
use serde::Deserialize;
|
||||
use thiserror::Error;
|
||||
use tracing::{debug, warn};
|
||||
|
||||
/// The api-v2 base. Every request also carries `client_id` (+ `app_version`).
|
||||
const API_BASE: &str = "https://api-v2.soundcloud.com";
|
||||
/// The web app, scraped for a `client_id`/`app_version`.
|
||||
const WEB_BASE: &str = "https://soundcloud.com";
|
||||
/// Max ids per `/tracks?ids=` hydration batch (SoundCloud's cap).
|
||||
const MAX_BATCH: usize = 50;
|
||||
|
||||
/// A typed SoundCloud request failure. Carries only non-secret context — never
|
||||
/// the `client_id`, the OAuth token, or a signed media URL.
|
||||
#[derive(Debug, Error)]
|
||||
pub enum FetchError {
|
||||
/// The HTTP call failed (transport, timeout, or non-success status).
|
||||
#[error("soundcloud request failed: {0}")]
|
||||
Http(String),
|
||||
/// The response body did not decode into the expected shape.
|
||||
#[error("soundcloud returned malformed data: {0}")]
|
||||
Decode(String),
|
||||
/// The resource does not exist (HTTP 404).
|
||||
#[error("soundcloud resource not found")]
|
||||
NotFound,
|
||||
/// Auth was rejected (HTTP 401/403) even after a `client_id` re-scrape,
|
||||
/// or an OAuth-only call without a token.
|
||||
#[error("soundcloud authentication failed")]
|
||||
Unauthorized,
|
||||
/// No usable `client_id` could be obtained (scrape failed and none was
|
||||
/// configured).
|
||||
#[error("soundcloud client_id unavailable")]
|
||||
NoClientId,
|
||||
/// The track exposes no HLS/mp3 transcoding we can play.
|
||||
#[error("soundcloud track is not streamable")]
|
||||
NotStreamable,
|
||||
}
|
||||
|
||||
/// A SoundCloud track. `id` is numeric (URL-safe). `duration_ms` is the track
|
||||
/// length in milliseconds. Transcoding selection is hidden inside the seam, so
|
||||
/// this domain type never carries stream URLs.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ScTrack {
|
||||
pub id: String,
|
||||
pub title: String,
|
||||
/// The uploader's display name (`user.username`).
|
||||
pub artist: String,
|
||||
pub duration_ms: Option<u32>,
|
||||
/// Album/release title from `publisher_metadata`, when present.
|
||||
pub album: Option<String>,
|
||||
}
|
||||
|
||||
/// A SoundCloud playlist/set. `tracks` may be **partially hydrated**: playlist
|
||||
/// detail can return track stubs (ids only), which the provider fills via
|
||||
/// [`Sc::hydrate_tracks`] in bounded batches.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ScPlaylist {
|
||||
pub id: String,
|
||||
pub title: String,
|
||||
pub artist: String,
|
||||
/// Track ids in playlist order (authoritative even when `tracks` is
|
||||
/// partial).
|
||||
pub track_ids: Vec<String>,
|
||||
/// Hydrated tracks known so far (a subset of `track_ids`, in order).
|
||||
pub tracks: Vec<ScTrack>,
|
||||
}
|
||||
|
||||
/// The result of resolving a permalink URL — a single track or a playlist.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Resolved {
|
||||
Track(ScTrack),
|
||||
Playlist(ScPlaylist),
|
||||
}
|
||||
|
||||
/// The SoundCloud operations the provider needs. Behind `Box<dyn Sc>` so tests
|
||||
/// fake it (architecture/soundcloud-provider.md D2).
|
||||
///
|
||||
/// Every method may transparently (re-)acquire a `client_id`; callers never
|
||||
/// pass one. Personal methods (`my_*`) require a configured OAuth token and
|
||||
/// return [`FetchError::Unauthorized`] without one.
|
||||
#[async_trait]
|
||||
pub trait Sc: Debug + Send + Sync {
|
||||
/// Resolves a permalink URL to a track or playlist.
|
||||
async fn resolve(&self, url: &str) -> Result<Resolved, FetchError>;
|
||||
|
||||
/// Tracks matching a free-text query (at most `limit`).
|
||||
async fn search_tracks(&self, query: &str, limit: usize) -> Result<Vec<ScTrack>, FetchError>;
|
||||
|
||||
/// Playlists matching a free-text query (at most `limit`).
|
||||
async fn search_playlists(
|
||||
&self,
|
||||
query: &str,
|
||||
limit: usize,
|
||||
) -> Result<Vec<ScPlaylist>, FetchError>;
|
||||
|
||||
/// One track's detail by id.
|
||||
async fn track_detail(&self, id: &str) -> Result<ScTrack, FetchError>;
|
||||
|
||||
/// One playlist's detail by id (tracks may be stubs — see [`ScPlaylist`]).
|
||||
async fn playlist_detail(&self, id: &str) -> Result<ScPlaylist, FetchError>;
|
||||
|
||||
/// Hydrates stub track ids into full tracks in ≤50-id batches; order
|
||||
/// follows `ids`.
|
||||
async fn hydrate_tracks(&self, ids: &[String]) -> Result<Vec<ScTrack>, FetchError>;
|
||||
|
||||
/// Resolves a track id to a **playable HLS `.m3u8` media URL**: picks the
|
||||
/// `hls + audio/mpeg` transcoding and exchanges its API URL for the signed
|
||||
/// media URL. The returned URL is ephemeral and secret — never logged.
|
||||
/// [`FetchError::NotStreamable`] when no such transcoding exists.
|
||||
async fn resolve_stream_url(&self, track_id: &str) -> Result<String, FetchError>;
|
||||
|
||||
/// The signed-in user's liked tracks (at most `limit`). Requires OAuth.
|
||||
async fn my_likes(&self, limit: usize) -> Result<Vec<ScTrack>, FetchError>;
|
||||
|
||||
/// The signed-in user's playlists (at most `limit`). Requires OAuth.
|
||||
async fn my_playlists(&self, limit: usize) -> Result<Vec<ScPlaylist>, FetchError>;
|
||||
}
|
||||
|
||||
/// Production `reqwest` client for `api-v2.soundcloud.com`. `Debug` redacts the
|
||||
/// `client_id` and OAuth token (hard rule: redact secrets from logs/reports).
|
||||
///
|
||||
/// The `client_id`/`app_version` are cached behind a lock so a 401 can swap in
|
||||
/// a freshly scraped id at runtime without rebuilding the client.
|
||||
pub struct ScApi {
|
||||
http: reqwest::Client,
|
||||
creds: tokio::sync::RwLock<Creds>,
|
||||
/// Optional OAuth bearer for personal calls. Secret — redacted.
|
||||
oauth_token: Option<String>,
|
||||
}
|
||||
|
||||
/// The mutable scraped credentials.
|
||||
#[derive(Default, Clone)]
|
||||
struct Creds {
|
||||
client_id: Option<String>,
|
||||
app_version: Option<String>,
|
||||
}
|
||||
|
||||
impl Debug for ScApi {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
f.debug_struct("ScApi")
|
||||
.field("client_id", &"<redacted>")
|
||||
.field(
|
||||
"oauth_token",
|
||||
&self.oauth_token.as_ref().map(|_| "<redacted>"),
|
||||
)
|
||||
.finish_non_exhaustive()
|
||||
}
|
||||
}
|
||||
|
||||
impl ScApi {
|
||||
/// Builds the client. `client_id` may be `None` (it is scraped on first
|
||||
/// use); `oauth_token` enables personal endpoints. `timeout` bounds every
|
||||
/// call (hard rule: timeouts on external calls).
|
||||
pub fn new(
|
||||
client_id: Option<String>,
|
||||
app_version: Option<String>,
|
||||
oauth_token: Option<String>,
|
||||
timeout: Duration,
|
||||
) -> Result<Self, FetchError> {
|
||||
let http = reqwest::Client::builder()
|
||||
.timeout(timeout)
|
||||
// A browser-ish UA: the web endpoints 403 obviously-bot clients.
|
||||
.user_agent(
|
||||
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 \
|
||||
(KHTML, like Gecko) Chrome/124.0 Safari/537.36",
|
||||
)
|
||||
.build()
|
||||
.map_err(|err| FetchError::Http(err.to_string()))?;
|
||||
Ok(Self {
|
||||
http,
|
||||
creds: tokio::sync::RwLock::new(Creds {
|
||||
client_id: client_id.filter(|s| !s.trim().is_empty()),
|
||||
app_version: app_version.filter(|s| !s.trim().is_empty()),
|
||||
}),
|
||||
oauth_token: oauth_token.filter(|s| !s.trim().is_empty()),
|
||||
})
|
||||
}
|
||||
|
||||
/// The currently cached `client_id`, if any (for `settings()` round-trip so
|
||||
/// a scraped id is persisted and not re-scraped every start, D5).
|
||||
pub async fn cached_client_id(&self) -> Option<String> {
|
||||
self.creds.read().await.client_id.clone()
|
||||
}
|
||||
|
||||
/// The currently cached `app_version`, if any (persisted like the id).
|
||||
pub async fn cached_app_version(&self) -> Option<String> {
|
||||
self.creds.read().await.app_version.clone()
|
||||
}
|
||||
|
||||
/// Ensures a `client_id` is available, scraping once if not, and returns a
|
||||
/// snapshot of the credentials. `init` calls this so a first run is ready
|
||||
/// (and so the scraped id can be persisted).
|
||||
pub async fn ensure_ready(&self) -> Result<(), FetchError> {
|
||||
let have = self.creds.read().await.client_id.is_some();
|
||||
if !have {
|
||||
self.rescrape().await?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Scrapes a fresh `client_id`/`app_version` and stores it under the write
|
||||
/// lock (never held across the network fetch: fetch first, then swap).
|
||||
async fn rescrape(&self) -> Result<(), FetchError> {
|
||||
let (client_id, app_version) = scrape_client_id(&self.http).await?;
|
||||
let mut creds = self.creds.write().await;
|
||||
creds.client_id = Some(client_id);
|
||||
creds.app_version = Some(app_version);
|
||||
debug!("soundcloud client_id refreshed");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A `client_id` snapshot, scraping if absent.
|
||||
async fn client_id(&self) -> Result<String, FetchError> {
|
||||
if let Some(id) = self.creds.read().await.client_id.clone() {
|
||||
return Ok(id);
|
||||
}
|
||||
self.rescrape().await?;
|
||||
self.creds
|
||||
.read()
|
||||
.await
|
||||
.client_id
|
||||
.clone()
|
||||
.ok_or(FetchError::NoClientId)
|
||||
}
|
||||
|
||||
/// Signed GET of `url` with the given query pairs, decoding JSON. On a
|
||||
/// 401/403 it re-scrapes the `client_id` **once** and retries; a second
|
||||
/// rejection is [`FetchError::Unauthorized`] (no loop, G8).
|
||||
async fn get_json<T: DeserializeOwned>(
|
||||
&self,
|
||||
url: &str,
|
||||
query: &[(&str, String)],
|
||||
) -> Result<T, FetchError> {
|
||||
match self.get_once(url, query).await {
|
||||
Err(FetchError::Unauthorized) => {
|
||||
warn!("soundcloud auth rejected; re-scraping client_id once");
|
||||
self.rescrape().await?;
|
||||
self.get_once(url, query).await
|
||||
}
|
||||
other => other,
|
||||
}
|
||||
}
|
||||
|
||||
async fn get_once<T: DeserializeOwned>(
|
||||
&self,
|
||||
url: &str,
|
||||
query: &[(&str, String)],
|
||||
) -> Result<T, FetchError> {
|
||||
let client_id = self.client_id().await?;
|
||||
let app_version = self.creds.read().await.app_version.clone();
|
||||
let mut req = self
|
||||
.http
|
||||
.get(url)
|
||||
.query(&[("client_id", client_id.as_str())]);
|
||||
if let Some(ver) = app_version.as_deref() {
|
||||
req = req.query(&[("app_version", ver)]);
|
||||
}
|
||||
if !query.is_empty() {
|
||||
req = req.query(query);
|
||||
}
|
||||
if let Some(token) = self.oauth_token.as_deref() {
|
||||
req = req.header(reqwest::header::AUTHORIZATION, format!("OAuth {token}"));
|
||||
}
|
||||
let resp = req
|
||||
.send()
|
||||
.await
|
||||
.map_err(|err| FetchError::Http(err.without_url().to_string()))?;
|
||||
match resp.status() {
|
||||
s if s.is_success() => resp
|
||||
.json::<T>()
|
||||
.await
|
||||
.map_err(|err| FetchError::Decode(err.to_string())),
|
||||
reqwest::StatusCode::NOT_FOUND => Err(FetchError::NotFound),
|
||||
reqwest::StatusCode::UNAUTHORIZED | reqwest::StatusCode::FORBIDDEN => {
|
||||
Err(FetchError::Unauthorized)
|
||||
}
|
||||
status => Err(FetchError::Http(format!("status {status}"))),
|
||||
}
|
||||
}
|
||||
|
||||
fn require_oauth(&self) -> Result<(), FetchError> {
|
||||
if self.oauth_token.is_some() {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(FetchError::Unauthorized)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl Sc for ScApi {
|
||||
async fn resolve(&self, url: &str) -> Result<Resolved, FetchError> {
|
||||
let dto: ResolveDto = self
|
||||
.get_json(&format!("{API_BASE}/resolve"), &[("url", url.to_string())])
|
||||
.await?;
|
||||
dto.into_resolved().ok_or(FetchError::NotFound)
|
||||
}
|
||||
|
||||
async fn search_tracks(&self, query: &str, limit: usize) -> Result<Vec<ScTrack>, FetchError> {
|
||||
let dto: Collection<TrackDto> = self
|
||||
.get_json(
|
||||
&format!("{API_BASE}/search/tracks"),
|
||||
&[
|
||||
("q", query.to_string()),
|
||||
("limit", limit.to_string()),
|
||||
("linked_partitioning", "1".to_string()),
|
||||
],
|
||||
)
|
||||
.await?;
|
||||
Ok(dto
|
||||
.collection
|
||||
.into_iter()
|
||||
.filter_map(TrackDto::into_track)
|
||||
.collect())
|
||||
}
|
||||
|
||||
async fn search_playlists(
|
||||
&self,
|
||||
query: &str,
|
||||
limit: usize,
|
||||
) -> Result<Vec<ScPlaylist>, FetchError> {
|
||||
let dto: Collection<PlaylistDto> = self
|
||||
.get_json(
|
||||
&format!("{API_BASE}/search/playlists"),
|
||||
&[
|
||||
("q", query.to_string()),
|
||||
("limit", limit.to_string()),
|
||||
("linked_partitioning", "1".to_string()),
|
||||
],
|
||||
)
|
||||
.await?;
|
||||
Ok(dto
|
||||
.collection
|
||||
.into_iter()
|
||||
.filter_map(PlaylistDto::into_playlist)
|
||||
.collect())
|
||||
}
|
||||
|
||||
async fn track_detail(&self, id: &str) -> Result<ScTrack, FetchError> {
|
||||
let dto: TrackDto = self
|
||||
.get_json(&format!("{API_BASE}/tracks/{id}"), &[])
|
||||
.await?;
|
||||
dto.into_track().ok_or(FetchError::NotFound)
|
||||
}
|
||||
|
||||
async fn playlist_detail(&self, id: &str) -> Result<ScPlaylist, FetchError> {
|
||||
let dto: PlaylistDto = self
|
||||
.get_json(&format!("{API_BASE}/playlists/{id}"), &[])
|
||||
.await?;
|
||||
dto.into_playlist().ok_or(FetchError::NotFound)
|
||||
}
|
||||
|
||||
async fn hydrate_tracks(&self, ids: &[String]) -> Result<Vec<ScTrack>, FetchError> {
|
||||
let mut out = Vec::with_capacity(ids.len());
|
||||
for batch in ids.chunks(MAX_BATCH) {
|
||||
let dto: Vec<TrackDto> = self
|
||||
.get_json(&format!("{API_BASE}/tracks"), &[("ids", batch.join(","))])
|
||||
.await?;
|
||||
out.extend(dto.into_iter().filter_map(TrackDto::into_track));
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
async fn resolve_stream_url(&self, track_id: &str) -> Result<String, FetchError> {
|
||||
let dto: TrackDto = self
|
||||
.get_json(&format!("{API_BASE}/tracks/{track_id}"), &[])
|
||||
.await?;
|
||||
let transcoding_url =
|
||||
pick_hls_mp3(&dto.media.transcodings).ok_or(FetchError::NotStreamable)?;
|
||||
let media: MediaUrl = self.get_json(&transcoding_url, &[]).await?;
|
||||
if media.url.is_empty() {
|
||||
return Err(FetchError::NotStreamable);
|
||||
}
|
||||
Ok(media.url)
|
||||
}
|
||||
|
||||
async fn my_likes(&self, limit: usize) -> Result<Vec<ScTrack>, FetchError> {
|
||||
self.require_oauth()?;
|
||||
let dto: Collection<LikeDto> = self
|
||||
.get_json(
|
||||
&format!("{API_BASE}/me/likes/tracks"),
|
||||
&[
|
||||
("limit", limit.to_string()),
|
||||
("linked_partitioning", "1".to_string()),
|
||||
],
|
||||
)
|
||||
.await?;
|
||||
Ok(dto
|
||||
.collection
|
||||
.into_iter()
|
||||
.filter_map(|l| l.track.and_then(TrackDto::into_track))
|
||||
.collect())
|
||||
}
|
||||
|
||||
async fn my_playlists(&self, limit: usize) -> Result<Vec<ScPlaylist>, FetchError> {
|
||||
self.require_oauth()?;
|
||||
let dto: Collection<PlaylistDto> = self
|
||||
.get_json(
|
||||
&format!("{API_BASE}/me/playlists"),
|
||||
&[
|
||||
("limit", limit.to_string()),
|
||||
("linked_partitioning", "1".to_string()),
|
||||
],
|
||||
)
|
||||
.await?;
|
||||
Ok(dto
|
||||
.collection
|
||||
.into_iter()
|
||||
.filter_map(PlaylistDto::into_playlist)
|
||||
.collect())
|
||||
}
|
||||
}
|
||||
|
||||
/// Picks the `hls + audio/mpeg` transcoding's API url (mp3-HLS, D4).
|
||||
fn pick_hls_mp3(transcodings: &[TranscodingDto]) -> Option<String> {
|
||||
transcodings
|
||||
.iter()
|
||||
.find(|t| t.format.protocol == "hls" && t.format.mime_type.starts_with("audio/mpeg"))
|
||||
.map(|t| t.url.clone())
|
||||
}
|
||||
|
||||
// --- client_id scraping: pure parsers + network glue -----------------------
|
||||
|
||||
/// Scrapes a fresh `(client_id, app_version)` from soundcloud.com. Fetches the
|
||||
/// homepage, then the referenced JS bundles (newest first — the `client_id`
|
||||
/// tends to live in the last one), returning the first match. Used at init
|
||||
/// when none is configured, and on a 401 to survive rotation (D5).
|
||||
pub async fn scrape_client_id(http: &reqwest::Client) -> Result<(String, String), FetchError> {
|
||||
let html = http
|
||||
.get(WEB_BASE)
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| FetchError::Http(e.without_url().to_string()))?
|
||||
.text()
|
||||
.await
|
||||
.map_err(|e| FetchError::Decode(e.to_string()))?;
|
||||
let app_version = extract_app_version(&html).unwrap_or_default();
|
||||
// Try the bundles in reverse: the client_id is usually in the last script.
|
||||
for url in extract_script_urls(&html).into_iter().rev() {
|
||||
let Ok(resp) = http.get(&url).send().await else {
|
||||
continue;
|
||||
};
|
||||
let Ok(js) = resp.text().await else { continue };
|
||||
if let Some(id) = extract_client_id(&js) {
|
||||
return Ok((id, app_version));
|
||||
}
|
||||
}
|
||||
Err(FetchError::NoClientId)
|
||||
}
|
||||
|
||||
/// Extracts `window.__sc_version="<digits>"`.
|
||||
fn extract_app_version(html: &str) -> Option<String> {
|
||||
let after = html.split_once("__sc_version=\"")?.1;
|
||||
let digits: String = after.chars().take_while(|c| c.is_ascii_digit()).collect();
|
||||
(!digits.is_empty()).then_some(digits)
|
||||
}
|
||||
|
||||
/// Extracts the `sndcdn.com/assets/*.js` bundle URLs referenced by the page, in
|
||||
/// document order.
|
||||
fn extract_script_urls(html: &str) -> Vec<String> {
|
||||
let mut urls = Vec::new();
|
||||
for part in html.split("src=\"").skip(1) {
|
||||
if let Some((url, _)) = part.split_once('"') {
|
||||
if url.contains("sndcdn.com") && url.ends_with(".js") {
|
||||
urls.push(url.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
urls
|
||||
}
|
||||
|
||||
/// Extracts `client_id:"<token>"` (or `client_id="..."`) from a JS bundle.
|
||||
fn extract_client_id(js: &str) -> Option<String> {
|
||||
for sep in ["client_id:\"", "client_id=\"", "\"client_id\":\""] {
|
||||
if let Some(after) = js.split_once(sep).map(|(_, r)| r) {
|
||||
let id: String = after
|
||||
.chars()
|
||||
.take_while(|c| c.is_ascii_alphanumeric())
|
||||
.collect();
|
||||
if id.len() >= 16 {
|
||||
return Some(id);
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
// --- Wire DTOs: decode defensively, missing fields degrade, never panic. ---
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct Collection<T> {
|
||||
#[serde(default = "Vec::new")]
|
||||
collection: Vec<T>,
|
||||
}
|
||||
|
||||
#[derive(Default, Deserialize)]
|
||||
struct UserDto {
|
||||
#[serde(default)]
|
||||
username: String,
|
||||
}
|
||||
|
||||
#[derive(Default, Deserialize)]
|
||||
struct PublisherDto {
|
||||
#[serde(default)]
|
||||
album_title: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Default, Deserialize)]
|
||||
struct FormatDto {
|
||||
#[serde(default)]
|
||||
protocol: String,
|
||||
#[serde(default)]
|
||||
mime_type: String,
|
||||
}
|
||||
|
||||
#[derive(Default, Deserialize)]
|
||||
struct TranscodingDto {
|
||||
#[serde(default)]
|
||||
url: String,
|
||||
#[serde(default)]
|
||||
format: FormatDto,
|
||||
}
|
||||
|
||||
#[derive(Default, Deserialize)]
|
||||
struct MediaDto {
|
||||
#[serde(default)]
|
||||
transcodings: Vec<TranscodingDto>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct MediaUrl {
|
||||
#[serde(default)]
|
||||
url: String,
|
||||
}
|
||||
|
||||
#[derive(Default, Deserialize)]
|
||||
struct TrackDto {
|
||||
#[serde(default)]
|
||||
id: Option<i64>,
|
||||
#[serde(default)]
|
||||
title: String,
|
||||
#[serde(default)]
|
||||
user: UserDto,
|
||||
#[serde(default)]
|
||||
duration: Option<u32>,
|
||||
#[serde(default)]
|
||||
publisher_metadata: Option<PublisherDto>,
|
||||
#[serde(default)]
|
||||
media: MediaDto,
|
||||
}
|
||||
|
||||
impl TrackDto {
|
||||
fn into_track(self) -> Option<ScTrack> {
|
||||
let id = self.id?.to_string();
|
||||
Some(ScTrack {
|
||||
id,
|
||||
title: self.title,
|
||||
artist: self.user.username,
|
||||
duration_ms: self.duration.filter(|d| *d > 0),
|
||||
album: self
|
||||
.publisher_metadata
|
||||
.and_then(|p| p.album_title)
|
||||
.filter(|s| !s.is_empty()),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct LikeDto {
|
||||
#[serde(default)]
|
||||
track: Option<TrackDto>,
|
||||
}
|
||||
|
||||
/// A playlist track can arrive as a full object or a `{id}` stub.
|
||||
#[derive(Default, Deserialize)]
|
||||
struct PlaylistTrackDto {
|
||||
#[serde(default)]
|
||||
id: Option<i64>,
|
||||
#[serde(default)]
|
||||
title: Option<String>,
|
||||
#[serde(default)]
|
||||
user: Option<UserDto>,
|
||||
#[serde(default)]
|
||||
duration: Option<u32>,
|
||||
}
|
||||
|
||||
#[derive(Default, Deserialize)]
|
||||
struct PlaylistDto {
|
||||
#[serde(default)]
|
||||
id: Option<i64>,
|
||||
#[serde(default)]
|
||||
title: String,
|
||||
#[serde(default)]
|
||||
user: UserDto,
|
||||
#[serde(default)]
|
||||
tracks: Vec<PlaylistTrackDto>,
|
||||
}
|
||||
|
||||
impl PlaylistDto {
|
||||
fn into_playlist(self) -> Option<ScPlaylist> {
|
||||
let id = self.id?.to_string();
|
||||
let mut track_ids = Vec::new();
|
||||
let mut tracks = Vec::new();
|
||||
for t in self.tracks {
|
||||
let Some(tid) = t.id else { continue };
|
||||
let tid = tid.to_string();
|
||||
// A track with a title is already hydrated; otherwise it is a stub.
|
||||
if let Some(title) = t.title {
|
||||
tracks.push(ScTrack {
|
||||
id: tid.clone(),
|
||||
title,
|
||||
artist: t.user.map(|u| u.username).unwrap_or_default(),
|
||||
duration_ms: t.duration.filter(|d| *d > 0),
|
||||
album: None,
|
||||
});
|
||||
}
|
||||
track_ids.push(tid);
|
||||
}
|
||||
Some(ScPlaylist {
|
||||
id,
|
||||
title: self.title,
|
||||
artist: self.user.username,
|
||||
track_ids,
|
||||
tracks,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// `/resolve` returns a track or a playlist, discriminated by `kind`.
|
||||
#[derive(Deserialize)]
|
||||
struct ResolveDto {
|
||||
#[serde(default)]
|
||||
kind: String,
|
||||
#[serde(flatten)]
|
||||
track: TrackDto,
|
||||
#[serde(default)]
|
||||
tracks: Vec<PlaylistTrackDto>,
|
||||
#[serde(default)]
|
||||
id: Option<i64>,
|
||||
#[serde(default)]
|
||||
title: String,
|
||||
#[serde(default)]
|
||||
user: UserDto,
|
||||
}
|
||||
|
||||
impl ResolveDto {
|
||||
fn into_resolved(self) -> Option<Resolved> {
|
||||
match self.kind.as_str() {
|
||||
"playlist" => PlaylistDto {
|
||||
id: self.id,
|
||||
title: self.title,
|
||||
user: self.user,
|
||||
tracks: self.tracks,
|
||||
}
|
||||
.into_playlist()
|
||||
.map(Resolved::Playlist),
|
||||
_ => self.track.into_track().map(Resolved::Track),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod api_tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn extracts_app_version() {
|
||||
let html = r#"<script>window.__sc_version="1700000000";</script>"#;
|
||||
assert_eq!(extract_app_version(html), Some("1700000000".to_string()));
|
||||
assert_eq!(extract_app_version("nope"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extracts_script_bundles_in_order() {
|
||||
let html = r#"
|
||||
<script crossorigin src="https://a-v2.sndcdn.com/assets/0-abc.js"></script>
|
||||
<script src="https://example.com/other.js"></script>
|
||||
<script crossorigin src="https://a-v2.sndcdn.com/assets/9-zzz.js"></script>
|
||||
"#;
|
||||
assert_eq!(
|
||||
extract_script_urls(html),
|
||||
vec![
|
||||
"https://a-v2.sndcdn.com/assets/0-abc.js".to_string(),
|
||||
"https://a-v2.sndcdn.com/assets/9-zzz.js".to_string(),
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extracts_client_id_from_js() {
|
||||
assert_eq!(
|
||||
extract_client_id(r#"...,client_id:"aBcDeF0123456789xyz",..."#),
|
||||
Some("aBcDeF0123456789xyz".to_string())
|
||||
);
|
||||
// Too-short tokens are ignored (avoids false positives).
|
||||
assert_eq!(extract_client_id(r#"client_id:"short""#), None);
|
||||
assert_eq!(extract_client_id("no id here"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn picks_hls_mp3_transcoding() {
|
||||
let transcodings = vec![
|
||||
TranscodingDto {
|
||||
url: "https://api/opus".into(),
|
||||
format: FormatDto {
|
||||
protocol: "hls".into(),
|
||||
mime_type: "audio/ogg; codecs=\"opus\"".into(),
|
||||
},
|
||||
},
|
||||
TranscodingDto {
|
||||
url: "https://api/mp3".into(),
|
||||
format: FormatDto {
|
||||
protocol: "hls".into(),
|
||||
mime_type: "audio/mpeg".into(),
|
||||
},
|
||||
},
|
||||
TranscodingDto {
|
||||
url: "https://api/prog".into(),
|
||||
format: FormatDto {
|
||||
protocol: "progressive".into(),
|
||||
mime_type: "audio/mpeg".into(),
|
||||
},
|
||||
},
|
||||
];
|
||||
assert_eq!(
|
||||
pick_hls_mp3(&transcodings),
|
||||
Some("https://api/mp3".to_string())
|
||||
);
|
||||
assert_eq!(pick_hls_mp3(&[]), None);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,651 @@
|
|||
//! SoundCloud provider: **search, resolve links, and play** SoundCloud tracks
|
||||
//! and playlists, plus — when an OAuth token is configured — the user's own
|
||||
//! likes and playlists. Mounted at [`PROVIDER_ROOT`].
|
||||
//!
|
||||
//! Shaped on the abs/fyyd providers (remote, search-driven), with two
|
||||
//! SoundCloud-specific twists (architecture/soundcloud-provider.md):
|
||||
//! 1. **Login is optional** — public browse/play needs only a scraped
|
||||
//! `client_id`; a token merely adds the `likes`/`playlists` nodes.
|
||||
//! 2. **Playback is HLS** — `get_urls_for_track` returns an `.m3u8` media URL
|
||||
//! that the audio-player's new `HlsStream` streams as continuous mp3.
|
||||
//!
|
||||
//! The `client_id`/OAuth token and the signed media URL are secrets, redacted
|
||||
//! from `Debug` and never logged.
|
||||
//!
|
||||
//! Provider tree/path logic (Phase A) is implemented and unit-tested over a
|
||||
//! fake; the `reqwest`-backed [`api::ScApi`] and its `init`/scrape flow are
|
||||
//! Phase B (see plan/soundcloud.md).
|
||||
|
||||
use std::fmt;
|
||||
use std::sync::RwLock;
|
||||
|
||||
use async_trait::async_trait;
|
||||
use crabidy_core::proto::crabidy::{Album, LibraryNode, LibraryNodeChild, Track};
|
||||
use crabidy_core::{ProviderClient, ProviderError};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use tracing::{debug, warn};
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
pub mod api;
|
||||
use api::{Resolved, Sc, ScApi, ScPlaylist, ScTrack};
|
||||
|
||||
/// First path segment owned by this provider.
|
||||
pub const PROVIDER_ROOT: &str = "/soundcloud";
|
||||
|
||||
/// Reserved second-level literals (numeric ids never collide with these).
|
||||
const SEARCH_SEGMENT: &str = "search";
|
||||
const RESOLVE_SEGMENT: &str = "resolve";
|
||||
const LIKES_SEGMENT: &str = "likes";
|
||||
const PLAYLISTS_SEGMENT: &str = "playlists";
|
||||
/// Canonical type tags disambiguating numeric track vs playlist ids.
|
||||
const TRACK_SEGMENT: &str = "track";
|
||||
const PLAYLIST_SEGMENT: &str = "playlist";
|
||||
|
||||
/// Default (and cap on) results per search term.
|
||||
pub const DEFAULT_SEARCH_RESULTS: usize = 50;
|
||||
/// Default (and cap on) tracks hydrated per playlist.
|
||||
pub const DEFAULT_PLAYLIST_TRACKS: usize = 500;
|
||||
/// Default per-request timeout in seconds.
|
||||
pub const DEFAULT_CALL_TIMEOUT_SECS: u64 = 30;
|
||||
/// Default total deadline for fetching a whole HLS stream, in seconds.
|
||||
pub const DEFAULT_HLS_DEADLINE_SECS: u64 = 300;
|
||||
|
||||
/// Provider settings, persisted as `soundcloud.toml`. All fields are optional:
|
||||
/// with none set, the provider still works (scrapes a `client_id`, no login).
|
||||
#[derive(Clone, Default, Deserialize, Serialize)]
|
||||
pub struct Settings {
|
||||
/// SoundCloud `client_id`. If unset it is scraped and then **written back
|
||||
/// here** by `init` so it persists. **Secret**: redacted from `Debug`.
|
||||
pub client_id: Option<String>,
|
||||
/// Web-app version paired with `client_id`; scraped and persisted like it.
|
||||
pub app_version: Option<String>,
|
||||
/// Optional OAuth token. Present → the `likes`/`playlists` nodes appear.
|
||||
/// **Secret**: redacted from `Debug`.
|
||||
pub oauth_token: Option<String>,
|
||||
/// Results per search term. Default [`DEFAULT_SEARCH_RESULTS`].
|
||||
pub search_results: Option<usize>,
|
||||
/// Tracks hydrated per playlist. Default [`DEFAULT_PLAYLIST_TRACKS`].
|
||||
pub playlist_tracks: Option<usize>,
|
||||
/// Per-request timeout in seconds. Default [`DEFAULT_CALL_TIMEOUT_SECS`].
|
||||
pub call_timeout_secs: Option<u64>,
|
||||
/// Total HLS-fetch deadline in seconds. Default [`DEFAULT_HLS_DEADLINE_SECS`].
|
||||
pub hls_deadline_secs: Option<u64>,
|
||||
}
|
||||
|
||||
impl fmt::Debug for Settings {
|
||||
/// Redacts `client_id` and `oauth_token` (hard rule: secrets never logged).
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
f.debug_struct("Settings")
|
||||
.field("client_id", &self.client_id.as_ref().map(|_| "<redacted>"))
|
||||
.field("app_version", &self.app_version)
|
||||
.field(
|
||||
"oauth_token",
|
||||
&self.oauth_token.as_ref().map(|_| "<redacted>"),
|
||||
)
|
||||
.field("search_results", &self.search_results)
|
||||
.field("playlist_tracks", &self.playlist_tracks)
|
||||
.field("call_timeout_secs", &self.call_timeout_secs)
|
||||
.field("hls_deadline_secs", &self.hls_deadline_secs)
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
/// A parsed `/soundcloud/...` path. Browse nodes (search/resolve/likes/
|
||||
/// playlists) point their children at the **canonical** `track/<id>` and
|
||||
/// `playlist/<id>` shapes, so a leaf carries the id alone — no browse context.
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
enum ScPath<'a> {
|
||||
Root,
|
||||
/// The search-terms parent (creatable).
|
||||
Search,
|
||||
/// A single search term (percent-encoded segment).
|
||||
SearchTerm {
|
||||
term: &'a str,
|
||||
},
|
||||
/// The resolve-URLs parent (creatable; a "term" is a pasted permalink).
|
||||
Resolve,
|
||||
/// One resolved permalink (percent-encoded URL segment).
|
||||
ResolveEntry {
|
||||
url: &'a str,
|
||||
},
|
||||
/// The signed-in user's liked tracks (login only).
|
||||
Likes,
|
||||
/// The signed-in user's playlists (login only).
|
||||
Playlists,
|
||||
/// A playlist container by id (canonical).
|
||||
Playlist {
|
||||
id: &'a str,
|
||||
},
|
||||
/// A track leaf by id (canonical).
|
||||
TrackLeaf {
|
||||
id: &'a str,
|
||||
},
|
||||
}
|
||||
|
||||
/// Splits a `/soundcloud/...` path into its recognized shape. Unknown shapes
|
||||
/// are [`ProviderError::MalformedPath`].
|
||||
fn parse_path(path: &str) -> Result<ScPath<'_>, ProviderError> {
|
||||
if path == PROVIDER_ROOT {
|
||||
return Ok(ScPath::Root);
|
||||
}
|
||||
let rest = path
|
||||
.strip_prefix("/soundcloud/")
|
||||
.ok_or(ProviderError::MalformedPath)?;
|
||||
let segments: Vec<&str> = rest.split('/').collect();
|
||||
if segments.iter().any(|segment| segment.is_empty()) {
|
||||
return Err(ProviderError::MalformedPath);
|
||||
}
|
||||
match segments.as_slice() {
|
||||
[s] if *s == SEARCH_SEGMENT => Ok(ScPath::Search),
|
||||
[s, term] if *s == SEARCH_SEGMENT => Ok(ScPath::SearchTerm { term }),
|
||||
[s] if *s == RESOLVE_SEGMENT => Ok(ScPath::Resolve),
|
||||
[s, url] if *s == RESOLVE_SEGMENT => Ok(ScPath::ResolveEntry { url }),
|
||||
[s] if *s == LIKES_SEGMENT => Ok(ScPath::Likes),
|
||||
[s] if *s == PLAYLISTS_SEGMENT => Ok(ScPath::Playlists),
|
||||
[s, id] if *s == PLAYLIST_SEGMENT => Ok(ScPath::Playlist { id }),
|
||||
[s, id] if *s == TRACK_SEGMENT => Ok(ScPath::TrackLeaf { id }),
|
||||
_ => Err(ProviderError::MalformedPath),
|
||||
}
|
||||
}
|
||||
|
||||
/// Maps a fetch failure to the trait-level error, logging the typed cause.
|
||||
fn fetch_err(context: &str, err: api::FetchError) -> ProviderError {
|
||||
warn!(context, "soundcloud fetch failed: {err}");
|
||||
ProviderError::FetchError
|
||||
}
|
||||
|
||||
/// Snapshot of an in-memory term/URL store (never held across awaits).
|
||||
fn store_snapshot(store: &RwLock<Vec<String>>) -> Vec<String> {
|
||||
store.read().map(|v| v.clone()).unwrap_or_default()
|
||||
}
|
||||
|
||||
/// Adds an item to a store if absent (dedup, creation order preserved).
|
||||
fn store_add(store: &RwLock<Vec<String>>, item: &str) {
|
||||
if let Ok(mut list) = store.write() {
|
||||
if !list.iter().any(|existing| existing == item) {
|
||||
list.push(item.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Removes an item; `true` when it existed.
|
||||
fn store_remove(store: &RwLock<Vec<String>>, item: &str) -> bool {
|
||||
match store.write() {
|
||||
Ok(mut list) => {
|
||||
let before = list.len();
|
||||
list.retain(|existing| existing != item);
|
||||
list.len() != before
|
||||
}
|
||||
Err(_) => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// The SoundCloud provider client.
|
||||
pub struct Client {
|
||||
api: Box<dyn Sc>,
|
||||
settings: Settings,
|
||||
/// Whether an OAuth token is configured (gates the personal nodes).
|
||||
logged_in: bool,
|
||||
/// Search terms created under `/soundcloud/search`, in creation order,
|
||||
/// deduplicated. In-memory only, never held across awaits.
|
||||
search_terms: RwLock<Vec<String>>,
|
||||
/// Resolved permalink URLs created under `/soundcloud/resolve`.
|
||||
resolve_urls: RwLock<Vec<String>>,
|
||||
}
|
||||
|
||||
impl fmt::Debug for Client {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
f.debug_struct("Client")
|
||||
.field("api", &self.api)
|
||||
.field("settings", &self.settings)
|
||||
.field("logged_in", &self.logged_in)
|
||||
.finish_non_exhaustive()
|
||||
}
|
||||
}
|
||||
|
||||
impl Client {
|
||||
/// A client over any [`Sc`] backend — the seam the tests use.
|
||||
fn with_api(api: Box<dyn Sc>, settings: Settings) -> Self {
|
||||
let logged_in = settings
|
||||
.oauth_token
|
||||
.as_deref()
|
||||
.is_some_and(|t| !t.trim().is_empty());
|
||||
Self {
|
||||
api,
|
||||
settings,
|
||||
logged_in,
|
||||
search_terms: RwLock::new(Vec::new()),
|
||||
resolve_urls: RwLock::new(Vec::new()),
|
||||
}
|
||||
}
|
||||
|
||||
fn search_limit(&self) -> usize {
|
||||
self.settings
|
||||
.search_results
|
||||
.unwrap_or(DEFAULT_SEARCH_RESULTS)
|
||||
}
|
||||
|
||||
fn playlist_limit(&self) -> usize {
|
||||
self.settings
|
||||
.playlist_tracks
|
||||
.unwrap_or(DEFAULT_PLAYLIST_TRACKS)
|
||||
}
|
||||
|
||||
// --- Node builders ---
|
||||
|
||||
/// The `/soundcloud` root: `search` + `resolve`, plus `likes`/`playlists`
|
||||
/// when [`Self::logged_in`]. Sync — the children are fixed (decision #1).
|
||||
fn root_node(&self) -> LibraryNode {
|
||||
let mut children = vec![
|
||||
creatable_child(join(PROVIDER_ROOT, SEARCH_SEGMENT), SEARCH_SEGMENT),
|
||||
creatable_child(join(PROVIDER_ROOT, RESOLVE_SEGMENT), RESOLVE_SEGMENT),
|
||||
];
|
||||
if self.logged_in {
|
||||
// `likes` holds tracks directly (queueable); `playlists` holds
|
||||
// playlist containers (not itself queueable).
|
||||
children.push(LibraryNodeChild::new(
|
||||
join(PROVIDER_ROOT, LIKES_SEGMENT),
|
||||
LIKES_SEGMENT.to_string(),
|
||||
true,
|
||||
));
|
||||
children.push(LibraryNodeChild::new(
|
||||
join(PROVIDER_ROOT, PLAYLISTS_SEGMENT),
|
||||
PLAYLISTS_SEGMENT.to_string(),
|
||||
false,
|
||||
));
|
||||
}
|
||||
LibraryNode {
|
||||
path: PROVIDER_ROOT.to_string(),
|
||||
title: "soundcloud".to_string(),
|
||||
parent: Some(crabidy_core::ROOT_PATH.to_string()),
|
||||
tracks: Vec::new(),
|
||||
children,
|
||||
is_queable: false,
|
||||
is_creatable: false,
|
||||
is_downloadable: false,
|
||||
tracks_deletable: false,
|
||||
is_captured: false,
|
||||
}
|
||||
}
|
||||
|
||||
/// The `/soundcloud/search` (or `/resolve`) node listing a store's entries,
|
||||
/// each editable + deletable. `title` names the node; entries link to
|
||||
/// `parent/<encoded>`.
|
||||
fn store_node(&self, path: &str, title: &str, store: &RwLock<Vec<String>>) -> LibraryNode {
|
||||
let children = store_snapshot(store)
|
||||
.iter()
|
||||
.map(|entry| LibraryNodeChild {
|
||||
is_editable: true,
|
||||
is_deletable: true,
|
||||
..LibraryNodeChild::new(
|
||||
join(path, &crabidy_core::encode_segment(entry)),
|
||||
entry.clone(),
|
||||
false,
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
LibraryNode {
|
||||
path: path.to_string(),
|
||||
title: title.to_string(),
|
||||
parent: Some(parent_of(path)),
|
||||
tracks: Vec::new(),
|
||||
children,
|
||||
is_queable: false,
|
||||
is_creatable: true,
|
||||
is_downloadable: false,
|
||||
tracks_deletable: false,
|
||||
is_captured: false,
|
||||
}
|
||||
}
|
||||
|
||||
/// A `/soundcloud/search/<term>` node: matching tracks as directly-playable
|
||||
/// rows (canonical paths) and matching playlists as queueable container
|
||||
/// children (G13).
|
||||
async fn search_term_node(&self, path: &str, term: &str) -> Result<LibraryNode, ProviderError> {
|
||||
let tracks = self
|
||||
.api
|
||||
.search_tracks(term, self.search_limit())
|
||||
.await
|
||||
.map_err(|err| fetch_err("search tracks", err))?;
|
||||
let playlists = self
|
||||
.api
|
||||
.search_playlists(term, self.search_limit())
|
||||
.await
|
||||
.map_err(|err| fetch_err("search playlists", err))?;
|
||||
Ok(self.list_node(path, term.to_string(), &tracks, &playlists))
|
||||
}
|
||||
|
||||
/// A `/soundcloud/resolve/<url>` node: the resolved track (single row) or
|
||||
/// playlist (a queueable container child).
|
||||
async fn resolve_entry_node(
|
||||
&self,
|
||||
path: &str,
|
||||
url: &str,
|
||||
) -> Result<LibraryNode, ProviderError> {
|
||||
match self
|
||||
.api
|
||||
.resolve(url)
|
||||
.await
|
||||
.map_err(|err| fetch_err("resolve", err))?
|
||||
{
|
||||
Resolved::Track(track) => Ok(self.list_node(path, url.to_string(), &[track], &[])),
|
||||
Resolved::Playlist(playlist) => {
|
||||
Ok(self.list_node(path, url.to_string(), &[], &[playlist]))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The `/soundcloud/likes` node: the user's liked tracks (login only).
|
||||
async fn likes_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
|
||||
let tracks = self
|
||||
.api
|
||||
.my_likes(self.search_limit())
|
||||
.await
|
||||
.map_err(|err| fetch_err("likes", err))?;
|
||||
Ok(self.list_node(path, LIKES_SEGMENT.to_string(), &tracks, &[]))
|
||||
}
|
||||
|
||||
/// The `/soundcloud/playlists` node: the user's playlists (login only).
|
||||
async fn playlists_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
|
||||
let playlists = self
|
||||
.api
|
||||
.my_playlists(self.search_limit())
|
||||
.await
|
||||
.map_err(|err| fetch_err("my playlists", err))?;
|
||||
Ok(self.list_node(path, PLAYLISTS_SEGMENT.to_string(), &[], &playlists))
|
||||
}
|
||||
|
||||
/// A `/soundcloud/playlist/<id>` node: the playlist's tracks, hydrating
|
||||
/// stub ids in bounded batches (G15).
|
||||
async fn playlist_node(&self, path: &str, id: &str) -> Result<LibraryNode, ProviderError> {
|
||||
let detail = self
|
||||
.api
|
||||
.playlist_detail(id)
|
||||
.await
|
||||
.map_err(|err| fetch_err("playlist detail", err))?;
|
||||
let limit = self.playlist_limit();
|
||||
if detail.track_ids.len() > limit {
|
||||
debug!(id, limit, "playlist truncated to the configured cap");
|
||||
}
|
||||
// Prefer tracks already inlined; hydrate the rest by id, in order.
|
||||
let tracks = if detail.tracks.len() >= detail.track_ids.len().min(limit) {
|
||||
detail.tracks
|
||||
} else {
|
||||
let ids: Vec<String> = detail.track_ids.iter().take(limit).cloned().collect();
|
||||
self.api
|
||||
.hydrate_tracks(&ids)
|
||||
.await
|
||||
.map_err(|err| fetch_err("hydrate tracks", err))?
|
||||
};
|
||||
Ok(self.list_node(path, detail.title, &tracks, &[]))
|
||||
}
|
||||
|
||||
/// Builds a node holding `tracks` as directly-playable rows (canonical
|
||||
/// paths) and `playlists` as queueable container children (canonical
|
||||
/// paths). Queueable when it carries either.
|
||||
fn list_node(
|
||||
&self,
|
||||
path: &str,
|
||||
title: String,
|
||||
tracks: &[ScTrack],
|
||||
playlists: &[ScPlaylist],
|
||||
) -> LibraryNode {
|
||||
let track_rows: Vec<Track> = tracks.iter().map(sc_track).collect();
|
||||
let children: Vec<LibraryNodeChild> = playlists
|
||||
.iter()
|
||||
.map(|p| {
|
||||
LibraryNodeChild::new(
|
||||
join(PROVIDER_ROOT, &format!("{PLAYLIST_SEGMENT}/{}", p.id)),
|
||||
p.title.clone(),
|
||||
true,
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
let is_queable = !track_rows.is_empty() || children.iter().any(|c| c.is_queable);
|
||||
LibraryNode {
|
||||
path: path.to_string(),
|
||||
title,
|
||||
parent: Some(parent_of(path)),
|
||||
tracks: track_rows,
|
||||
children,
|
||||
is_queable,
|
||||
is_creatable: false,
|
||||
is_downloadable: false,
|
||||
tracks_deletable: false,
|
||||
is_captured: false,
|
||||
}
|
||||
}
|
||||
|
||||
/// The track id addressed by a canonical track path, or `MalformedPath`.
|
||||
fn track_id<'a>(&self, path: &'a str) -> Result<&'a str, ProviderError> {
|
||||
match parse_path(path)? {
|
||||
ScPath::TrackLeaf { id } => Ok(id),
|
||||
_ => Err(ProviderError::MalformedPath),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Builds the wire [`Track`] for a SoundCloud track at its canonical path.
|
||||
/// Duration is milliseconds→seconds; `provider_item_id` is `"track:<id>"`.
|
||||
fn sc_track(track: &ScTrack) -> Track {
|
||||
Track {
|
||||
path: join(PROVIDER_ROOT, &format!("{TRACK_SEGMENT}/{}", track.id)),
|
||||
artist: track.artist.clone(),
|
||||
title: track.title.clone(),
|
||||
duration: track.duration_ms.map(|ms| ms / 1000),
|
||||
album: track.album.as_ref().map(|title| Album {
|
||||
title: title.clone(),
|
||||
release_date: None,
|
||||
}),
|
||||
is_skipped: false,
|
||||
provider_item_id: format!("track:{}", track.id),
|
||||
is_captured: false,
|
||||
}
|
||||
}
|
||||
|
||||
/// A creatable browse child (search/resolve parents).
|
||||
fn creatable_child(path: String, title: &str) -> LibraryNodeChild {
|
||||
LibraryNodeChild {
|
||||
is_creatable: true,
|
||||
..LibraryNodeChild::new(path, title.to_string(), false)
|
||||
}
|
||||
}
|
||||
|
||||
fn join(base: &str, segment: &str) -> String {
|
||||
crabidy_core::join_path(base, segment)
|
||||
}
|
||||
|
||||
fn parent_of(path: &str) -> String {
|
||||
crabidy_core::parent_path(path)
|
||||
.unwrap_or(crabidy_core::ROOT_PATH)
|
||||
.to_string()
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl ProviderClient for Client {
|
||||
/// Parses `soundcloud.toml`, builds the `reqwest`-backed [`api::ScApi`]
|
||||
/// (scraping a `client_id` if none configured), and enables the personal
|
||||
/// nodes when an OAuth token is present. Phase B (see plan/soundcloud.md).
|
||||
async fn init(raw_toml_settings: &str) -> Result<Self, ProviderError> {
|
||||
let mut settings: Settings = toml::from_str(raw_toml_settings).unwrap_or_else(|_| {
|
||||
warn!("could not parse soundcloud.toml, using defaults");
|
||||
Settings::default()
|
||||
});
|
||||
let timeout = Duration::from_secs(
|
||||
settings
|
||||
.call_timeout_secs
|
||||
.unwrap_or(DEFAULT_CALL_TIMEOUT_SECS),
|
||||
);
|
||||
let api = ScApi::new(
|
||||
settings.client_id.clone(),
|
||||
settings.app_version.clone(),
|
||||
settings.oauth_token.clone(),
|
||||
timeout,
|
||||
)
|
||||
.map_err(|err| {
|
||||
warn!("cannot build the soundcloud client: {err}");
|
||||
ProviderError::Config(err.to_string())
|
||||
})?;
|
||||
// Ensure a client_id (scrape if none). A total failure disables the
|
||||
// provider non-fatally (D5/G7).
|
||||
api.ensure_ready().await.map_err(|err| {
|
||||
warn!("soundcloud provider disabled: no client_id ({err})");
|
||||
ProviderError::Config("soundcloud client_id unavailable".to_string())
|
||||
})?;
|
||||
// Persist the (possibly freshly scraped) credentials so the next start
|
||||
// does not re-scrape (D5/G9).
|
||||
settings.client_id = api.cached_client_id().await;
|
||||
settings.app_version = api.cached_app_version().await;
|
||||
debug!("soundcloud provider ready");
|
||||
Ok(Self::with_api(Box::new(api), settings))
|
||||
}
|
||||
|
||||
/// Serializes settings back to TOML. `init` folds a freshly scraped
|
||||
/// `client_id`/`app_version` into `self.settings`, so this persists them.
|
||||
fn settings(&self) -> String {
|
||||
toml::to_string_pretty(&self.settings).unwrap_or_default()
|
||||
}
|
||||
|
||||
fn is_track_path(&self, path: &str) -> bool {
|
||||
matches!(parse_path(path), Ok(ScPath::TrackLeaf { .. }))
|
||||
}
|
||||
|
||||
/// Resolves the track id to a signed HLS `.m3u8` media URL (one API round
|
||||
/// trip). The URL is ephemeral and secret — never logged (D4).
|
||||
async fn get_urls_for_track(&self, track_path: &str) -> Result<Vec<String>, ProviderError> {
|
||||
let id = self.track_id(track_path)?;
|
||||
let url = self
|
||||
.api
|
||||
.resolve_stream_url(id)
|
||||
.await
|
||||
.map_err(|err| fetch_err("stream url", err))?;
|
||||
Ok(vec![url])
|
||||
}
|
||||
|
||||
async fn get_metadata_for_track(&self, track_path: &str) -> Result<Track, ProviderError> {
|
||||
let id = self.track_id(track_path)?;
|
||||
let track = self
|
||||
.api
|
||||
.track_detail(id)
|
||||
.await
|
||||
.map_err(|err| fetch_err("track metadata", err))?;
|
||||
let mut wire = sc_track(&track);
|
||||
wire.path = track_path.to_string();
|
||||
Ok(wire)
|
||||
}
|
||||
|
||||
/// The root is served synchronously (its children are fixed).
|
||||
fn get_lib_root(&self) -> LibraryNode {
|
||||
self.root_node()
|
||||
}
|
||||
|
||||
async fn get_lib_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
|
||||
let mut node = match parse_path(path)? {
|
||||
ScPath::Root => self.root_node(),
|
||||
ScPath::Search => self.store_node(path, SEARCH_SEGMENT, &self.search_terms),
|
||||
ScPath::SearchTerm { term } => {
|
||||
let decoded = crabidy_core::decode_segment(term);
|
||||
// Unknown terms (stale client cache) are recreated implicitly.
|
||||
store_add(&self.search_terms, &decoded);
|
||||
self.search_term_node(path, &decoded).await?
|
||||
}
|
||||
ScPath::Resolve => self.store_node(path, RESOLVE_SEGMENT, &self.resolve_urls),
|
||||
ScPath::ResolveEntry { url } => {
|
||||
let decoded = crabidy_core::decode_segment(url);
|
||||
store_add(&self.resolve_urls, &decoded);
|
||||
self.resolve_entry_node(path, &decoded).await?
|
||||
}
|
||||
ScPath::Likes if self.logged_in => self.likes_node(path).await?,
|
||||
ScPath::Playlists if self.logged_in => self.playlists_node(path).await?,
|
||||
ScPath::Likes | ScPath::Playlists => {
|
||||
warn!(path, "personal node requested without login");
|
||||
return Err(ProviderError::MalformedPath);
|
||||
}
|
||||
ScPath::Playlist { id } => self.playlist_node(path, id).await?,
|
||||
ScPath::TrackLeaf { .. } => {
|
||||
warn!(path, "get_lib_node called with a track path");
|
||||
return Err(ProviderError::MalformedPath);
|
||||
}
|
||||
};
|
||||
// Download blessing (same rule as abs/fyyd): a node serving tracks and
|
||||
// each queueable child allows `W`.
|
||||
node.is_downloadable = node.is_queable || !node.tracks.is_empty();
|
||||
for child in &mut node.children {
|
||||
child.is_downloadable = child.is_queable;
|
||||
}
|
||||
Ok(node)
|
||||
}
|
||||
|
||||
/// `/soundcloud/search` and `/soundcloud/resolve` are creatable: register a
|
||||
/// term / permalink and return its node (implicit recreation on stale
|
||||
/// paths, like the other providers).
|
||||
async fn create_lib_node(
|
||||
&self,
|
||||
parent_path: &str,
|
||||
title: &str,
|
||||
) -> Result<LibraryNode, ProviderError> {
|
||||
let entry = title.trim();
|
||||
if entry.is_empty() {
|
||||
return Err(ProviderError::InvalidInput);
|
||||
}
|
||||
let (store, base) = match parse_path(parent_path)? {
|
||||
ScPath::Search => (&self.search_terms, SEARCH_SEGMENT),
|
||||
ScPath::Resolve => (&self.resolve_urls, RESOLVE_SEGMENT),
|
||||
_ => {
|
||||
warn!(parent_path, "node creation not supported here");
|
||||
return Err(ProviderError::NotSupported);
|
||||
}
|
||||
};
|
||||
store_add(store, entry);
|
||||
let entry_path = join(
|
||||
&join(PROVIDER_ROOT, base),
|
||||
&crabidy_core::encode_segment(entry),
|
||||
);
|
||||
self.get_lib_node(&entry_path).await
|
||||
}
|
||||
|
||||
async fn rename_lib_node(
|
||||
&self,
|
||||
path: &str,
|
||||
new_title: &str,
|
||||
) -> Result<LibraryNode, ProviderError> {
|
||||
let (store, base, old_encoded) = match parse_path(path)? {
|
||||
ScPath::SearchTerm { term } => (&self.search_terms, SEARCH_SEGMENT, term),
|
||||
ScPath::ResolveEntry { url } => (&self.resolve_urls, RESOLVE_SEGMENT, url),
|
||||
_ => {
|
||||
warn!(path, "only search terms / resolve urls are renamable");
|
||||
return Err(ProviderError::NotSupported);
|
||||
}
|
||||
};
|
||||
let new_entry = new_title.trim();
|
||||
if new_entry.is_empty() {
|
||||
return Err(ProviderError::InvalidInput);
|
||||
}
|
||||
store_remove(store, &crabidy_core::decode_segment(old_encoded));
|
||||
store_add(store, new_entry);
|
||||
let new_path = join(
|
||||
&join(PROVIDER_ROOT, base),
|
||||
&crabidy_core::encode_segment(new_entry),
|
||||
);
|
||||
self.get_lib_node(&new_path).await
|
||||
}
|
||||
|
||||
async fn delete_lib_node(&self, path: &str) -> Result<LibraryNode, ProviderError> {
|
||||
let (store, base, encoded) = match parse_path(path)? {
|
||||
ScPath::SearchTerm { term } => (&self.search_terms, SEARCH_SEGMENT, term),
|
||||
ScPath::ResolveEntry { url } => (&self.resolve_urls, RESOLVE_SEGMENT, url),
|
||||
_ => {
|
||||
warn!(path, "only search terms / resolve urls are deletable");
|
||||
return Err(ProviderError::NotSupported);
|
||||
}
|
||||
};
|
||||
store_remove(store, &crabidy_core::decode_segment(encoded));
|
||||
self.get_lib_node(&join(PROVIDER_ROOT, base)).await
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
include!("tests.rs");
|
||||
}
|
||||
|
|
@ -0,0 +1,283 @@
|
|||
// Unit tests over a fixture-driven `FakeSc` backend (no network). Path parsing
|
||||
// and secret redaction are real now; the behavioural tests define the Stage-5
|
||||
// target and fail (on `todo!()`) until the provider logic is implemented.
|
||||
|
||||
use std::collections::HashMap;
|
||||
|
||||
use super::*;
|
||||
use api::{Resolved, ScPlaylist, ScTrack};
|
||||
|
||||
// --- Pure logic (passes today) -------------------------------------------
|
||||
|
||||
#[test]
|
||||
fn root_path_parses() {
|
||||
assert_eq!(parse_path("/soundcloud"), Ok(ScPath::Root));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn canonical_leaves_parse() {
|
||||
assert_eq!(parse_path("/soundcloud/track/12345"), Ok(ScPath::TrackLeaf { id: "12345" }));
|
||||
assert_eq!(parse_path("/soundcloud/playlist/999"), Ok(ScPath::Playlist { id: "999" }));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn browse_parents_parse() {
|
||||
assert_eq!(parse_path("/soundcloud/search"), Ok(ScPath::Search));
|
||||
assert_eq!(parse_path("/soundcloud/resolve"), Ok(ScPath::Resolve));
|
||||
assert_eq!(parse_path("/soundcloud/likes"), Ok(ScPath::Likes));
|
||||
assert_eq!(parse_path("/soundcloud/playlists"), Ok(ScPath::Playlists));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn term_and_url_segments_parse() {
|
||||
assert_eq!(
|
||||
parse_path("/soundcloud/search/boards%20of%20canada"),
|
||||
Ok(ScPath::SearchTerm { term: "boards%20of%20canada" })
|
||||
);
|
||||
assert!(matches!(
|
||||
parse_path("/soundcloud/resolve/https%3A%2F%2Fsoundcloud.com%2Fx"),
|
||||
Ok(ScPath::ResolveEntry { .. })
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn foreign_and_malformed_paths_reject() {
|
||||
assert_eq!(parse_path("/tidal/x"), Err(ProviderError::MalformedPath));
|
||||
assert_eq!(parse_path("/soundcloud/"), Err(ProviderError::MalformedPath));
|
||||
assert_eq!(parse_path("/soundcloud/track"), Err(ProviderError::MalformedPath));
|
||||
assert_eq!(parse_path("/soundcloud/track/1/2"), Err(ProviderError::MalformedPath));
|
||||
assert_eq!(parse_path("/soundcloud//x"), Err(ProviderError::MalformedPath));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn is_track_path_only_for_canonical_track() {
|
||||
let client = Client::with_api(Box::new(FakeSc::default()), Settings::default());
|
||||
assert!(client.is_track_path("/soundcloud/track/1"));
|
||||
assert!(!client.is_track_path("/soundcloud/playlist/1"));
|
||||
assert!(!client.is_track_path("/soundcloud/search/x"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn settings_debug_redacts_secrets() {
|
||||
let s = Settings {
|
||||
client_id: Some("SECRET_ID".into()),
|
||||
oauth_token: Some("SECRET_TOKEN".into()),
|
||||
..Settings::default()
|
||||
};
|
||||
let dumped = format!("{s:?}");
|
||||
assert!(!dumped.contains("SECRET_ID"));
|
||||
assert!(!dumped.contains("SECRET_TOKEN"));
|
||||
assert!(dumped.contains("<redacted>"));
|
||||
}
|
||||
|
||||
// --- Behavioural (Stage-5 target; fails on `todo!()` until implemented) ---
|
||||
|
||||
#[tokio::test]
|
||||
async fn root_without_login_has_no_personal_nodes() {
|
||||
let client = Client::with_api(Box::new(FakeSc::default()), Settings::default());
|
||||
let root = client.get_lib_root();
|
||||
let paths: Vec<&str> = root.children.iter().map(|c| c.path.as_str()).collect();
|
||||
assert!(paths.contains(&"/soundcloud/search"));
|
||||
assert!(paths.contains(&"/soundcloud/resolve"));
|
||||
assert!(!paths.iter().any(|p| p.ends_with("/likes") || p.ends_with("/playlists")));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn root_with_login_shows_personal_nodes() {
|
||||
let settings = Settings { oauth_token: Some("tok".into()), ..Settings::default() };
|
||||
let client = Client::with_api(Box::new(FakeSc::default()), settings);
|
||||
let paths: Vec<String> = client.get_lib_root().children.iter().map(|c| c.path.clone()).collect();
|
||||
assert!(paths.iter().any(|p| p == "/soundcloud/likes"));
|
||||
assert!(paths.iter().any(|p| p == "/soundcloud/playlists"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn search_term_lists_tracks_and_playlists_with_canonical_paths() {
|
||||
let fake = FakeSc {
|
||||
search_tracks: vec![track("10", "Kaini Industries")],
|
||||
search_playlists: vec![playlist("77", "BoC set", &["10"])],
|
||||
..FakeSc::default()
|
||||
};
|
||||
let client = Client::with_api(Box::new(fake), Settings::default());
|
||||
let node = client.get_lib_node("/soundcloud/search/boc").await.unwrap();
|
||||
// Tracks are directly-playable rows on the node, at canonical paths.
|
||||
assert!(node.tracks.iter().any(|t| t.path == "/soundcloud/track/10"));
|
||||
// Playlists are queueable container children, at canonical paths.
|
||||
let child = node.children.iter().find(|c| c.path == "/soundcloud/playlist/77");
|
||||
assert!(child.is_some(), "playlist child missing");
|
||||
assert!(child.unwrap().is_queable);
|
||||
assert!(node.is_queable);
|
||||
assert!(node.is_downloadable, "download blessing");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn get_urls_for_track_returns_the_m3u8() {
|
||||
let fake = FakeSc {
|
||||
streams: HashMap::from([("10".to_string(), "https://cf-hls.sndcdn.com/x.m3u8".to_string())]),
|
||||
..FakeSc::default()
|
||||
};
|
||||
let client = Client::with_api(Box::new(fake), Settings::default());
|
||||
let urls = client.get_urls_for_track("/soundcloud/track/10").await.unwrap();
|
||||
assert_eq!(urls, vec!["https://cf-hls.sndcdn.com/x.m3u8".to_string()]);
|
||||
// A non-track path is rejected, not streamed.
|
||||
assert_eq!(
|
||||
client.get_urls_for_track("/soundcloud/playlist/77").await,
|
||||
Err(ProviderError::MalformedPath)
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn resolve_entry_shapes_track_and_playlist() {
|
||||
let fake = FakeSc {
|
||||
resolve: HashMap::from([
|
||||
("https://soundcloud.com/a/t".to_string(), Resolved::Track(track("10", "T"))),
|
||||
("https://soundcloud.com/a/set".to_string(), Resolved::Playlist(playlist("77", "S", &["10"]))),
|
||||
]),
|
||||
..FakeSc::default()
|
||||
};
|
||||
let client = Client::with_api(Box::new(fake), Settings::default());
|
||||
client.create_lib_node("/soundcloud/resolve", "https://soundcloud.com/a/t").await.unwrap();
|
||||
let enc = crabidy_core::encode_segment("https://soundcloud.com/a/t");
|
||||
let node = client.get_lib_node(&format!("/soundcloud/resolve/{enc}")).await.unwrap();
|
||||
assert!(node.tracks.iter().any(|t| t.path == "/soundcloud/track/10"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn playlist_node_hydrates_stub_tracks() {
|
||||
let fake = FakeSc {
|
||||
playlists: HashMap::from([("77".to_string(), playlist("77", "S", &["10", "11"]))]),
|
||||
tracks: HashMap::from([
|
||||
("10".to_string(), track("10", "A")),
|
||||
("11".to_string(), track("11", "B")),
|
||||
]),
|
||||
..FakeSc::default()
|
||||
};
|
||||
let client = Client::with_api(Box::new(fake), Settings::default());
|
||||
let node = client.get_lib_node("/soundcloud/playlist/77").await.unwrap();
|
||||
assert_eq!(node.tracks.len(), 2);
|
||||
assert!(node.is_queable && node.is_downloadable);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn create_rename_delete_search_term() {
|
||||
let client = Client::with_api(Box::new(FakeSc::default()), Settings::default());
|
||||
client.create_lib_node("/soundcloud/search", "aphex").await.unwrap();
|
||||
let search = client.get_lib_node("/soundcloud/search").await.unwrap();
|
||||
assert!(search.children.iter().any(|c| c.title == "aphex" && c.is_deletable && c.is_editable));
|
||||
// Empty title is rejected.
|
||||
assert_eq!(
|
||||
client.create_lib_node("/soundcloud/search", " ").await,
|
||||
Err(ProviderError::InvalidInput)
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn backend_failures_are_typed() {
|
||||
// A backend that always fails must surface FetchError, never panic.
|
||||
let client = Client::with_api(Box::new(FailingSc), Settings::default());
|
||||
assert_eq!(
|
||||
client.get_lib_node("/soundcloud/search/x").await,
|
||||
Err(ProviderError::FetchError)
|
||||
);
|
||||
}
|
||||
|
||||
// --- Fakes and fixtures ---------------------------------------------------
|
||||
|
||||
fn track(id: &str, title: &str) -> ScTrack {
|
||||
ScTrack {
|
||||
id: id.into(),
|
||||
title: title.into(),
|
||||
artist: "Boards of Canada".into(),
|
||||
duration_ms: Some(180_000),
|
||||
album: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn playlist(id: &str, title: &str, track_ids: &[&str]) -> ScPlaylist {
|
||||
ScPlaylist {
|
||||
id: id.into(),
|
||||
title: title.into(),
|
||||
artist: "Boards of Canada".into(),
|
||||
track_ids: track_ids.iter().map(|s| s.to_string()).collect(),
|
||||
tracks: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// A fixture-driven fake: every method reads a field, defaulting to empty.
|
||||
#[derive(Debug, Default)]
|
||||
struct FakeSc {
|
||||
tracks: HashMap<String, ScTrack>,
|
||||
playlists: HashMap<String, ScPlaylist>,
|
||||
search_tracks: Vec<ScTrack>,
|
||||
search_playlists: Vec<ScPlaylist>,
|
||||
resolve: HashMap<String, Resolved>,
|
||||
likes: Vec<ScTrack>,
|
||||
my_playlists: Vec<ScPlaylist>,
|
||||
streams: HashMap<String, String>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl api::Sc for FakeSc {
|
||||
async fn resolve(&self, url: &str) -> Result<Resolved, api::FetchError> {
|
||||
self.resolve.get(url).cloned().ok_or(api::FetchError::NotFound)
|
||||
}
|
||||
async fn search_tracks(&self, _q: &str, _l: usize) -> Result<Vec<ScTrack>, api::FetchError> {
|
||||
Ok(self.search_tracks.clone())
|
||||
}
|
||||
async fn search_playlists(&self, _q: &str, _l: usize) -> Result<Vec<ScPlaylist>, api::FetchError> {
|
||||
Ok(self.search_playlists.clone())
|
||||
}
|
||||
async fn track_detail(&self, id: &str) -> Result<ScTrack, api::FetchError> {
|
||||
self.tracks.get(id).cloned().ok_or(api::FetchError::NotFound)
|
||||
}
|
||||
async fn playlist_detail(&self, id: &str) -> Result<ScPlaylist, api::FetchError> {
|
||||
self.playlists.get(id).cloned().ok_or(api::FetchError::NotFound)
|
||||
}
|
||||
async fn hydrate_tracks(&self, ids: &[String]) -> Result<Vec<ScTrack>, api::FetchError> {
|
||||
Ok(ids.iter().filter_map(|id| self.tracks.get(id).cloned()).collect())
|
||||
}
|
||||
async fn resolve_stream_url(&self, id: &str) -> Result<String, api::FetchError> {
|
||||
self.streams.get(id).cloned().ok_or(api::FetchError::NotStreamable)
|
||||
}
|
||||
async fn my_likes(&self, _l: usize) -> Result<Vec<ScTrack>, api::FetchError> {
|
||||
Ok(self.likes.clone())
|
||||
}
|
||||
async fn my_playlists(&self, _l: usize) -> Result<Vec<ScPlaylist>, api::FetchError> {
|
||||
Ok(self.my_playlists.clone())
|
||||
}
|
||||
}
|
||||
|
||||
/// A backend whose every call fails — for the no-panic / typed-error gate.
|
||||
#[derive(Debug)]
|
||||
struct FailingSc;
|
||||
|
||||
#[async_trait]
|
||||
impl api::Sc for FailingSc {
|
||||
async fn resolve(&self, _u: &str) -> Result<Resolved, api::FetchError> {
|
||||
Err(api::FetchError::Http("boom".into()))
|
||||
}
|
||||
async fn search_tracks(&self, _q: &str, _l: usize) -> Result<Vec<ScTrack>, api::FetchError> {
|
||||
Err(api::FetchError::Http("boom".into()))
|
||||
}
|
||||
async fn search_playlists(&self, _q: &str, _l: usize) -> Result<Vec<ScPlaylist>, api::FetchError> {
|
||||
Err(api::FetchError::Http("boom".into()))
|
||||
}
|
||||
async fn track_detail(&self, _id: &str) -> Result<ScTrack, api::FetchError> {
|
||||
Err(api::FetchError::Http("boom".into()))
|
||||
}
|
||||
async fn playlist_detail(&self, _id: &str) -> Result<ScPlaylist, api::FetchError> {
|
||||
Err(api::FetchError::Http("boom".into()))
|
||||
}
|
||||
async fn hydrate_tracks(&self, _ids: &[String]) -> Result<Vec<ScTrack>, api::FetchError> {
|
||||
Err(api::FetchError::Http("boom".into()))
|
||||
}
|
||||
async fn resolve_stream_url(&self, _id: &str) -> Result<String, api::FetchError> {
|
||||
Err(api::FetchError::Http("boom".into()))
|
||||
}
|
||||
async fn my_likes(&self, _l: usize) -> Result<Vec<ScTrack>, api::FetchError> {
|
||||
Err(api::FetchError::Http("boom".into()))
|
||||
}
|
||||
async fn my_playlists(&self, _l: usize) -> Result<Vec<ScPlaylist>, api::FetchError> {
|
||||
Err(api::FetchError::Http("boom".into()))
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
//! Live validation against the real SoundCloud API. `#[ignore]`d so it never
|
||||
//! runs in CI or hits the network by default; run it deliberately with
|
||||
//!
|
||||
//! ```sh
|
||||
//! cargo test -p soundclouddy --test live -- --ignored --nocapture
|
||||
//! # optional: SOUNDCLOUD_CLIENT_ID=<id> to skip the scrape,
|
||||
//! # SOUNDCLOUD_OAUTH=<token> to also exercise the personal calls.
|
||||
//! ```
|
||||
//!
|
||||
//! It exercises the real endpoints end-to-end (scrape client_id → search →
|
||||
//! resolve stream URL) so the `ScApi` DTOs and the `client_id` scrape are
|
||||
//! confirmed against the live shapes — the drift risk called out in
|
||||
//! architecture/soundcloud-provider.md.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
use soundclouddy::api::{Sc, ScApi};
|
||||
|
||||
fn api() -> ScApi {
|
||||
ScApi::new(
|
||||
std::env::var("SOUNDCLOUD_CLIENT_ID").ok(),
|
||||
None,
|
||||
std::env::var("SOUNDCLOUD_OAUTH").ok(),
|
||||
Duration::from_secs(30),
|
||||
)
|
||||
.expect("client builds")
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "hits the real SoundCloud API (scrapes a client_id)"]
|
||||
async fn live_scrape_search_and_stream() {
|
||||
let api = api();
|
||||
// Scrapes a client_id if none was provided; a failure here means the
|
||||
// scrape parsers need updating against the current soundcloud.com.
|
||||
api.ensure_ready().await.expect("obtain a client_id");
|
||||
println!(
|
||||
"client_id acquired: {}",
|
||||
api.cached_client_id().await.is_some()
|
||||
);
|
||||
|
||||
let tracks = api
|
||||
.search_tracks("boards of canada", 5)
|
||||
.await
|
||||
.expect("search decodes");
|
||||
assert!(!tracks.is_empty(), "search returns tracks");
|
||||
let first = &tracks[0];
|
||||
println!(
|
||||
"first track: {} — {} ({})",
|
||||
first.artist, first.title, first.id
|
||||
);
|
||||
|
||||
// Resolve a playable HLS media URL for it.
|
||||
let url = api
|
||||
.resolve_stream_url(&first.id)
|
||||
.await
|
||||
.expect("stream url resolves");
|
||||
assert!(
|
||||
url.contains(".m3u8"),
|
||||
"stream URL is an HLS playlist: {url}"
|
||||
);
|
||||
println!("stream url resolves to an m3u8 (len {})", url.len());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[ignore = "hits the real SoundCloud API; requires SOUNDCLOUD_OAUTH"]
|
||||
async fn live_personal_likes() {
|
||||
if std::env::var("SOUNDCLOUD_OAUTH").is_err() {
|
||||
eprintln!("SOUNDCLOUD_OAUTH unset — skipping personal test");
|
||||
return;
|
||||
}
|
||||
let api = api();
|
||||
api.ensure_ready().await.expect("client_id");
|
||||
let likes = api.my_likes(5).await.expect("likes decode");
|
||||
println!("liked tracks: {}", likes.len());
|
||||
}
|
||||
Loading…
Reference in New Issue