soundcloud: prefer the progressive mp3 stream (HLS exchange 404s anonymously)
Live testing showed SoundCloud only serves the progressive+audio/mpeg transcoding to anonymous clients: its media exchange returns 200 with a direct, range-streamable mp3 URL (cf-media.sndcdn.com, 206, audio/mpeg), while the plain hls+audio/mpeg exchange 404s for every track (streamable or not). The provider picked HLS, so every queued track failed to resolve a URL and was skipped. pick_stream_url now prefers progressive, falling back to hls. A progressive URL is a plain mp3 the player streams on its normal windowed-HTTP path (no .m3u8, so HlsStream is bypassed); HlsStream stays the fallback for HLS-only tracks. A genuinely restricted track (Go+/label preview, geo-blocked) still 404s the exchange and is skipped, not crashed. Note: SoundCloud login does not help here - public streaming is client_id-only. Verified live: search a streamable track -> resolve -> 206 range GET returns audio/mpeg with an mp3 frame-sync header. 19 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
6abda3aa58
commit
739c5a805a
|
|
@ -135,14 +135,24 @@ Track ids and playlist ids are both numeric, so leaf/container paths use a
|
||||||
Terms/URLs are percent-encoded into one segment (`encode_segment`/
|
Terms/URLs are percent-encoded into one segment (`encode_segment`/
|
||||||
`decode_segment`); numeric ids are already URL-safe.
|
`decode_segment`); numeric ids are already URL-safe.
|
||||||
|
|
||||||
### D4 — Playback via a new HLS source in `audio-player`
|
### D4 — Playback: progressive mp3 (HLS source retained as fallback)
|
||||||
|
|
||||||
- `get_urls_for_track(/soundcloud/track/<id>)`: `GET /tracks/<id>` (or use a
|
**Revised after live testing (2026-07-24).** The original plan chose HLS mp3,
|
||||||
cached transcoding), pick the `hls + audio/mpeg` transcoding, `GET
|
but live probing showed SoundCloud's plain `hls + audio/mpeg` transcoding
|
||||||
<transcoding.url>?client_id=…` → the `.m3u8` media URL, and **return that URL
|
exchange **404s for anonymous clients** (every track, streamable or not), while
|
||||||
as `urls[0]`** (the player consumes only the first). One API round-trip —
|
the **`progressive + audio/mpeg`** transcoding returns 200 with a direct,
|
||||||
unlike abs's pure string-building — because the media URL is signed and
|
range-streamable mp3 URL (`cf-media.sndcdn.com`, `206`, `audio/mpeg`). So the
|
||||||
ephemeral.
|
provider now **prefers progressive**, which the player streams on its normal
|
||||||
|
windowed-HTTP path — no HLS needed for the common case. The `HlsStream` built
|
||||||
|
for the original plan is kept as a fallback for any track that offers only HLS.
|
||||||
|
|
||||||
|
- `get_urls_for_track(/soundcloud/track/<id>)`: `GET /tracks/<id>`, pick the
|
||||||
|
best mp3 transcoding (`progressive` first, then `hls`), `GET
|
||||||
|
<transcoding.url>?client_id=…` → the media URL, and **return it as `urls[0]`**
|
||||||
|
(the player consumes only the first). One API round-trip — unlike abs's pure
|
||||||
|
string-building — because the media URL is signed and ephemeral. A track
|
||||||
|
whose exchange 404s (Go+/label preview, geo-blocked) surfaces
|
||||||
|
`NotStreamable`/`NotFound` and is skipped, never a crash.
|
||||||
- **New `audio-player` component `HlsStream`** — a `stream-download`
|
- **New `audio-player` component `HlsStream`** — a `stream-download`
|
||||||
`SourceStream`, sibling to `WindowedHttpStream`: on create it fetches the
|
`SourceStream`, sibling to `WindowedHttpStream`: on create it fetches the
|
||||||
`.m3u8` (a media playlist), parses `#EXTINF`/segment URIs (resolving relative
|
`.m3u8` (a media playlist), parses `#EXTINF`/segment URIs (resolving relative
|
||||||
|
|
|
||||||
|
|
@ -122,10 +122,11 @@ pub trait Sc: Debug + Send + Sync {
|
||||||
/// follows `ids`.
|
/// follows `ids`.
|
||||||
async fn hydrate_tracks(&self, ids: &[String]) -> Result<Vec<ScTrack>, FetchError>;
|
async fn hydrate_tracks(&self, ids: &[String]) -> Result<Vec<ScTrack>, FetchError>;
|
||||||
|
|
||||||
/// Resolves a track id to a **playable HLS `.m3u8` media URL**: picks the
|
/// Resolves a track id to a **playable, signed media URL**: picks the best
|
||||||
/// `hls + audio/mpeg` transcoding and exchanges its API URL for the signed
|
/// mp3 transcoding (progressive preferred — it streams anonymously; HLS
|
||||||
/// media URL. The returned URL is ephemeral and secret — never logged.
|
/// `.m3u8` as fallback) and exchanges its API URL for the media URL. The
|
||||||
/// [`FetchError::NotStreamable`] when no such transcoding exists.
|
/// returned URL is ephemeral and secret — never logged.
|
||||||
|
/// [`FetchError::NotStreamable`] when no mp3 transcoding exists.
|
||||||
async fn resolve_stream_url(&self, track_id: &str) -> Result<String, FetchError>;
|
async fn resolve_stream_url(&self, track_id: &str) -> Result<String, FetchError>;
|
||||||
|
|
||||||
/// The signed-in user's liked tracks (at most `limit`). Requires OAuth.
|
/// The signed-in user's liked tracks (at most `limit`). Requires OAuth.
|
||||||
|
|
@ -385,7 +386,7 @@ impl Sc for ScApi {
|
||||||
.get_json(&format!("{API_BASE}/tracks/{track_id}"), &[])
|
.get_json(&format!("{API_BASE}/tracks/{track_id}"), &[])
|
||||||
.await?;
|
.await?;
|
||||||
let transcoding_url =
|
let transcoding_url =
|
||||||
pick_hls_mp3(&dto.media.transcodings).ok_or(FetchError::NotStreamable)?;
|
pick_stream_url(&dto.media.transcodings).ok_or(FetchError::NotStreamable)?;
|
||||||
let media: MediaUrl = self.get_json(&transcoding_url, &[]).await?;
|
let media: MediaUrl = self.get_json(&transcoding_url, &[]).await?;
|
||||||
if media.url.is_empty() {
|
if media.url.is_empty() {
|
||||||
return Err(FetchError::NotStreamable);
|
return Err(FetchError::NotStreamable);
|
||||||
|
|
@ -430,11 +431,23 @@ impl Sc for ScApi {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Picks the `hls + audio/mpeg` transcoding's API url (mp3-HLS, D4).
|
/// Picks the best playable mp3 transcoding's API url. **Prefers `progressive`
|
||||||
fn pick_hls_mp3(transcodings: &[TranscodingDto]) -> Option<String> {
|
/// over `hls`**: SoundCloud anonymously serves the progressive mp3 stream (a
|
||||||
|
/// direct, range-streamable file the player decodes on the normal path), while
|
||||||
|
/// the plain `hls` mp3 exchange currently 404s for anonymous clients (verified
|
||||||
|
/// live 2026-07-24). HLS stays a fallback for the rare track that only offers
|
||||||
|
/// it (then the player's `HlsStream` kicks in on the `.m3u8`). The
|
||||||
|
/// `*-encrypted-hls` variants are AES/DRM and deliberately ignored.
|
||||||
|
fn pick_stream_url(transcodings: &[TranscodingDto]) -> Option<String> {
|
||||||
|
let is_mp3 = |t: &&TranscodingDto| t.format.mime_type.starts_with("audio/mpeg");
|
||||||
transcodings
|
transcodings
|
||||||
.iter()
|
.iter()
|
||||||
.find(|t| t.format.protocol == "hls" && t.format.mime_type.starts_with("audio/mpeg"))
|
.find(|t| t.format.protocol == "progressive" && is_mp3(t))
|
||||||
|
.or_else(|| {
|
||||||
|
transcodings
|
||||||
|
.iter()
|
||||||
|
.find(|t| t.format.protocol == "hls" && is_mp3(t))
|
||||||
|
})
|
||||||
.map(|t| t.url.clone())
|
.map(|t| t.url.clone())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -524,7 +537,7 @@ struct PublisherDto {
|
||||||
album_title: Option<String>,
|
album_title: Option<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Default, Deserialize)]
|
#[derive(Default, Clone, Deserialize)]
|
||||||
struct FormatDto {
|
struct FormatDto {
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
protocol: String,
|
protocol: String,
|
||||||
|
|
@ -532,7 +545,7 @@ struct FormatDto {
|
||||||
mime_type: String,
|
mime_type: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Default, Deserialize)]
|
#[derive(Default, Clone, Deserialize)]
|
||||||
struct TranscodingDto {
|
struct TranscodingDto {
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
url: String,
|
url: String,
|
||||||
|
|
@ -717,34 +730,39 @@ mod api_tests {
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn picks_hls_mp3_transcoding() {
|
fn prefers_progressive_then_hls_mp3() {
|
||||||
let transcodings = vec![
|
let opus_hls = TranscodingDto {
|
||||||
TranscodingDto {
|
url: "https://api/opus".into(),
|
||||||
url: "https://api/opus".into(),
|
format: FormatDto {
|
||||||
format: FormatDto {
|
protocol: "hls".into(),
|
||||||
protocol: "hls".into(),
|
mime_type: "audio/ogg; codecs=\"opus\"".into(),
|
||||||
mime_type: "audio/ogg; codecs=\"opus\"".into(),
|
|
||||||
},
|
|
||||||
},
|
},
|
||||||
TranscodingDto {
|
};
|
||||||
url: "https://api/mp3".into(),
|
let mp3_hls = TranscodingDto {
|
||||||
format: FormatDto {
|
url: "https://api/mp3-hls".into(),
|
||||||
protocol: "hls".into(),
|
format: FormatDto {
|
||||||
mime_type: "audio/mpeg".into(),
|
protocol: "hls".into(),
|
||||||
},
|
mime_type: "audio/mpeg".into(),
|
||||||
},
|
},
|
||||||
TranscodingDto {
|
};
|
||||||
url: "https://api/prog".into(),
|
let mp3_prog = TranscodingDto {
|
||||||
format: FormatDto {
|
url: "https://api/mp3-prog".into(),
|
||||||
protocol: "progressive".into(),
|
format: FormatDto {
|
||||||
mime_type: "audio/mpeg".into(),
|
protocol: "progressive".into(),
|
||||||
},
|
mime_type: "audio/mpeg".into(),
|
||||||
},
|
},
|
||||||
];
|
};
|
||||||
|
// Progressive wins when present (it is what streams anonymously).
|
||||||
|
let all = vec![opus_hls, mp3_hls.clone(), mp3_prog];
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
pick_hls_mp3(&transcodings),
|
pick_stream_url(&all),
|
||||||
Some("https://api/mp3".to_string())
|
Some("https://api/mp3-prog".to_string())
|
||||||
);
|
);
|
||||||
assert_eq!(pick_hls_mp3(&[]), None);
|
// HLS mp3 is the fallback when there is no progressive.
|
||||||
|
assert_eq!(
|
||||||
|
pick_stream_url(&[mp3_hls]),
|
||||||
|
Some("https://api/mp3-hls".to_string())
|
||||||
|
);
|
||||||
|
assert_eq!(pick_stream_url(&[]), None);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -38,10 +38,10 @@ async fn live_scrape_search_and_stream() {
|
||||||
api.cached_client_id().await.is_some()
|
api.cached_client_id().await.is_some()
|
||||||
);
|
);
|
||||||
|
|
||||||
let tracks = api
|
// "lofi" returns freely-streamable (policy=MONETIZE) tracks; a label
|
||||||
.search_tracks("boards of canada", 5)
|
// artist like Boards of Canada would be Go+/preview-only and 404 the media
|
||||||
.await
|
// exchange (that restriction is expected, not a provider bug).
|
||||||
.expect("search decodes");
|
let tracks = api.search_tracks("lofi", 5).await.expect("search decodes");
|
||||||
assert!(!tracks.is_empty(), "search returns tracks");
|
assert!(!tracks.is_empty(), "search returns tracks");
|
||||||
let first = &tracks[0];
|
let first = &tracks[0];
|
||||||
println!(
|
println!(
|
||||||
|
|
@ -49,16 +49,13 @@ async fn live_scrape_search_and_stream() {
|
||||||
first.artist, first.title, first.id
|
first.artist, first.title, first.id
|
||||||
);
|
);
|
||||||
|
|
||||||
// Resolve a playable HLS media URL for it.
|
// Resolve a playable media URL (progressive mp3, or an HLS m3u8 fallback).
|
||||||
let url = api
|
let url = api
|
||||||
.resolve_stream_url(&first.id)
|
.resolve_stream_url(&first.id)
|
||||||
.await
|
.await
|
||||||
.expect("stream url resolves");
|
.expect("stream url resolves");
|
||||||
assert!(
|
assert!(url.starts_with("http"), "stream URL is absolute: {url}");
|
||||||
url.contains(".m3u8"),
|
println!("stream url resolves (len {})", url.len());
|
||||||
"stream URL is an HLS playlist: {url}"
|
|
||||||
);
|
|
||||||
println!("stream url resolves to an m3u8 (len {})", url.len());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[tokio::test]
|
#[tokio::test]
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue