soundcloud: prefer the progressive mp3 stream (HLS exchange 404s anonymously)

Live testing showed SoundCloud only serves the progressive+audio/mpeg
transcoding to anonymous clients: its media exchange returns 200 with a direct,
range-streamable mp3 URL (cf-media.sndcdn.com, 206, audio/mpeg), while the plain
hls+audio/mpeg exchange 404s for every track (streamable or not). The provider
picked HLS, so every queued track failed to resolve a URL and was skipped.

pick_stream_url now prefers progressive, falling back to hls. A progressive URL
is a plain mp3 the player streams on its normal windowed-HTTP path (no .m3u8, so
HlsStream is bypassed); HlsStream stays the fallback for HLS-only tracks. A
genuinely restricted track (Go+/label preview, geo-blocked) still 404s the
exchange and is skipped, not crashed. Note: SoundCloud login does not help here
- public streaming is client_id-only.

Verified live: search a streamable track -> resolve -> 206 range GET returns
audio/mpeg with an mp3 frame-sync header. 19 unit tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Test User 2026-07-24 02:47:13 +02:00
parent 6abda3aa58
commit 739c5a805a
3 changed files with 76 additions and 51 deletions

View File

@ -135,14 +135,24 @@ Track ids and playlist ids are both numeric, so leaf/container paths use a
Terms/URLs are percent-encoded into one segment (`encode_segment`/ Terms/URLs are percent-encoded into one segment (`encode_segment`/
`decode_segment`); numeric ids are already URL-safe. `decode_segment`); numeric ids are already URL-safe.
### D4 — Playback via a new HLS source in `audio-player` ### D4 — Playback: progressive mp3 (HLS source retained as fallback)
- `get_urls_for_track(/soundcloud/track/<id>)`: `GET /tracks/<id>` (or use a **Revised after live testing (2026-07-24).** The original plan chose HLS mp3,
cached transcoding), pick the `hls + audio/mpeg` transcoding, `GET but live probing showed SoundCloud's plain `hls + audio/mpeg` transcoding
<transcoding.url>?client_id=…` → the `.m3u8` media URL, and **return that URL exchange **404s for anonymous clients** (every track, streamable or not), while
as `urls[0]`** (the player consumes only the first). One API round-trip — the **`progressive + audio/mpeg`** transcoding returns 200 with a direct,
unlike abs's pure string-building — because the media URL is signed and range-streamable mp3 URL (`cf-media.sndcdn.com`, `206`, `audio/mpeg`). So the
ephemeral. provider now **prefers progressive**, which the player streams on its normal
windowed-HTTP path — no HLS needed for the common case. The `HlsStream` built
for the original plan is kept as a fallback for any track that offers only HLS.
- `get_urls_for_track(/soundcloud/track/<id>)`: `GET /tracks/<id>`, pick the
best mp3 transcoding (`progressive` first, then `hls`), `GET
<transcoding.url>?client_id=…` → the media URL, and **return it as `urls[0]`**
(the player consumes only the first). One API round-trip — unlike abs's pure
string-building — because the media URL is signed and ephemeral. A track
whose exchange 404s (Go+/label preview, geo-blocked) surfaces
`NotStreamable`/`NotFound` and is skipped, never a crash.
- **New `audio-player` component `HlsStream`** — a `stream-download` - **New `audio-player` component `HlsStream`** — a `stream-download`
`SourceStream`, sibling to `WindowedHttpStream`: on create it fetches the `SourceStream`, sibling to `WindowedHttpStream`: on create it fetches the
`.m3u8` (a media playlist), parses `#EXTINF`/segment URIs (resolving relative `.m3u8` (a media playlist), parses `#EXTINF`/segment URIs (resolving relative

View File

@ -122,10 +122,11 @@ pub trait Sc: Debug + Send + Sync {
/// follows `ids`. /// follows `ids`.
async fn hydrate_tracks(&self, ids: &[String]) -> Result<Vec<ScTrack>, FetchError>; async fn hydrate_tracks(&self, ids: &[String]) -> Result<Vec<ScTrack>, FetchError>;
/// Resolves a track id to a **playable HLS `.m3u8` media URL**: picks the /// Resolves a track id to a **playable, signed media URL**: picks the best
/// `hls + audio/mpeg` transcoding and exchanges its API URL for the signed /// mp3 transcoding (progressive preferred — it streams anonymously; HLS
/// media URL. The returned URL is ephemeral and secret — never logged. /// `.m3u8` as fallback) and exchanges its API URL for the media URL. The
/// [`FetchError::NotStreamable`] when no such transcoding exists. /// returned URL is ephemeral and secret — never logged.
/// [`FetchError::NotStreamable`] when no mp3 transcoding exists.
async fn resolve_stream_url(&self, track_id: &str) -> Result<String, FetchError>; async fn resolve_stream_url(&self, track_id: &str) -> Result<String, FetchError>;
/// The signed-in user's liked tracks (at most `limit`). Requires OAuth. /// The signed-in user's liked tracks (at most `limit`). Requires OAuth.
@ -385,7 +386,7 @@ impl Sc for ScApi {
.get_json(&format!("{API_BASE}/tracks/{track_id}"), &[]) .get_json(&format!("{API_BASE}/tracks/{track_id}"), &[])
.await?; .await?;
let transcoding_url = let transcoding_url =
pick_hls_mp3(&dto.media.transcodings).ok_or(FetchError::NotStreamable)?; pick_stream_url(&dto.media.transcodings).ok_or(FetchError::NotStreamable)?;
let media: MediaUrl = self.get_json(&transcoding_url, &[]).await?; let media: MediaUrl = self.get_json(&transcoding_url, &[]).await?;
if media.url.is_empty() { if media.url.is_empty() {
return Err(FetchError::NotStreamable); return Err(FetchError::NotStreamable);
@ -430,11 +431,23 @@ impl Sc for ScApi {
} }
} }
/// Picks the `hls + audio/mpeg` transcoding's API url (mp3-HLS, D4). /// Picks the best playable mp3 transcoding's API url. **Prefers `progressive`
fn pick_hls_mp3(transcodings: &[TranscodingDto]) -> Option<String> { /// over `hls`**: SoundCloud anonymously serves the progressive mp3 stream (a
/// direct, range-streamable file the player decodes on the normal path), while
/// the plain `hls` mp3 exchange currently 404s for anonymous clients (verified
/// live 2026-07-24). HLS stays a fallback for the rare track that only offers
/// it (then the player's `HlsStream` kicks in on the `.m3u8`). The
/// `*-encrypted-hls` variants are AES/DRM and deliberately ignored.
fn pick_stream_url(transcodings: &[TranscodingDto]) -> Option<String> {
let is_mp3 = |t: &&TranscodingDto| t.format.mime_type.starts_with("audio/mpeg");
transcodings transcodings
.iter() .iter()
.find(|t| t.format.protocol == "hls" && t.format.mime_type.starts_with("audio/mpeg")) .find(|t| t.format.protocol == "progressive" && is_mp3(t))
.or_else(|| {
transcodings
.iter()
.find(|t| t.format.protocol == "hls" && is_mp3(t))
})
.map(|t| t.url.clone()) .map(|t| t.url.clone())
} }
@ -524,7 +537,7 @@ struct PublisherDto {
album_title: Option<String>, album_title: Option<String>,
} }
#[derive(Default, Deserialize)] #[derive(Default, Clone, Deserialize)]
struct FormatDto { struct FormatDto {
#[serde(default)] #[serde(default)]
protocol: String, protocol: String,
@ -532,7 +545,7 @@ struct FormatDto {
mime_type: String, mime_type: String,
} }
#[derive(Default, Deserialize)] #[derive(Default, Clone, Deserialize)]
struct TranscodingDto { struct TranscodingDto {
#[serde(default)] #[serde(default)]
url: String, url: String,
@ -717,34 +730,39 @@ mod api_tests {
} }
#[test] #[test]
fn picks_hls_mp3_transcoding() { fn prefers_progressive_then_hls_mp3() {
let transcodings = vec![ let opus_hls = TranscodingDto {
TranscodingDto { url: "https://api/opus".into(),
url: "https://api/opus".into(), format: FormatDto {
format: FormatDto { protocol: "hls".into(),
protocol: "hls".into(), mime_type: "audio/ogg; codecs=\"opus\"".into(),
mime_type: "audio/ogg; codecs=\"opus\"".into(),
},
}, },
TranscodingDto { };
url: "https://api/mp3".into(), let mp3_hls = TranscodingDto {
format: FormatDto { url: "https://api/mp3-hls".into(),
protocol: "hls".into(), format: FormatDto {
mime_type: "audio/mpeg".into(), protocol: "hls".into(),
}, mime_type: "audio/mpeg".into(),
}, },
TranscodingDto { };
url: "https://api/prog".into(), let mp3_prog = TranscodingDto {
format: FormatDto { url: "https://api/mp3-prog".into(),
protocol: "progressive".into(), format: FormatDto {
mime_type: "audio/mpeg".into(), protocol: "progressive".into(),
}, mime_type: "audio/mpeg".into(),
}, },
]; };
// Progressive wins when present (it is what streams anonymously).
let all = vec![opus_hls, mp3_hls.clone(), mp3_prog];
assert_eq!( assert_eq!(
pick_hls_mp3(&transcodings), pick_stream_url(&all),
Some("https://api/mp3".to_string()) Some("https://api/mp3-prog".to_string())
); );
assert_eq!(pick_hls_mp3(&[]), None); // HLS mp3 is the fallback when there is no progressive.
assert_eq!(
pick_stream_url(&[mp3_hls]),
Some("https://api/mp3-hls".to_string())
);
assert_eq!(pick_stream_url(&[]), None);
} }
} }

View File

@ -38,10 +38,10 @@ async fn live_scrape_search_and_stream() {
api.cached_client_id().await.is_some() api.cached_client_id().await.is_some()
); );
let tracks = api // "lofi" returns freely-streamable (policy=MONETIZE) tracks; a label
.search_tracks("boards of canada", 5) // artist like Boards of Canada would be Go+/preview-only and 404 the media
.await // exchange (that restriction is expected, not a provider bug).
.expect("search decodes"); let tracks = api.search_tracks("lofi", 5).await.expect("search decodes");
assert!(!tracks.is_empty(), "search returns tracks"); assert!(!tracks.is_empty(), "search returns tracks");
let first = &tracks[0]; let first = &tracks[0];
println!( println!(
@ -49,16 +49,13 @@ async fn live_scrape_search_and_stream() {
first.artist, first.title, first.id first.artist, first.title, first.id
); );
// Resolve a playable HLS media URL for it. // Resolve a playable media URL (progressive mp3, or an HLS m3u8 fallback).
let url = api let url = api
.resolve_stream_url(&first.id) .resolve_stream_url(&first.id)
.await .await
.expect("stream url resolves"); .expect("stream url resolves");
assert!( assert!(url.starts_with("http"), "stream URL is absolute: {url}");
url.contains(".m3u8"), println!("stream url resolves (len {})", url.len());
"stream URL is an HLS playlist: {url}"
);
println!("stream url resolves to an m3u8 (len {})", url.len());
} }
#[tokio::test] #[tokio::test]