101 lines
4.3 KiB
Markdown
101 lines
4.3 KiB
Markdown
# Frequency spectrum visualizer
|
||
|
||
A row of frequency bars under the track progress in the TUI, on by
|
||
default, turn-offable in the client config. The reference the user gave,
|
||
[BeSpec](https://github.com/BeSpec-Dev/BeSpec), is a standalone
|
||
egui/wgpu app that captures *local* system-audio loopback and runs a
|
||
2048-point realfft — we borrow its DSP shape, not its capture model.
|
||
|
||
## The core problem: where does the audio live?
|
||
|
||
The audio is decoded and played **on the server** (the `audio-player`
|
||
crate, rodio). The TUI — and the web client — are gRPC clients that may
|
||
run on another machine (the stated setup: `cbd` local, `cbd-tui`
|
||
pointed at a Raspberry Pi, `architecture/client-configs.md`). A local
|
||
loopback capture in the client, BeSpec-style, would therefore show
|
||
nothing (or the wrong machine's audio) for a remote client.
|
||
|
||
So the spectrum must be produced where the samples are — the server —
|
||
and streamed to clients like every other bit of live state.
|
||
|
||
## Decisions
|
||
|
||
### D1 — server taps the samples, computes the FFT, streams bins
|
||
|
||
- **Tap** (`audio-player`): the decoded source is wrapped in a
|
||
`TappingSource` that copies each played frame (downmixed to mono)
|
||
into a fixed lock-free ring (`SpectrumTap`, 2048 `f32` slots, atomic
|
||
write index). It runs on rodio's audio thread, so it does the
|
||
absolute minimum — one store per sample, no locks, no allocation —
|
||
and benign read/write races are fine for a visualizer.
|
||
- **FFT** (`crabidy-server`): a task ticks at ~20 fps, snapshots the
|
||
ring, applies a Hann window + realfft, folds the magnitude spectrum
|
||
into a small number of log-spaced bins (musically even), normalizes
|
||
to `0..1`, and broadcasts them.
|
||
- **Stream**: a new `SpectrumFrame { bins }` on the existing
|
||
`GetUpdateStream` (oneof variant), at a handful of bins × ~20 fps —
|
||
~2 KB/s, negligible next to the audio it describes.
|
||
|
||
Rejected: client-side loopback capture (breaks the remote client, the
|
||
whole reason clients exist); sending raw PCM to clients (orders of
|
||
magnitude more bandwidth, and every client would re-run the FFT).
|
||
|
||
### D2 — idle detection without touching the control path
|
||
|
||
The tap increments a frame counter on every write. The FFT task
|
||
compares the counter between ticks: advancing ⇒ audio is flowing, emit
|
||
bins; unchanged ⇒ paused/stopped/between tracks, emit a single
|
||
all-zero frame (bars fall to the floor) and then stay quiet until it
|
||
moves again. No extra `is_playing` round-trips onto the player command
|
||
channel, and no frozen bars on pause.
|
||
|
||
### D3 — the config toggle is client-side display; the server always offers it
|
||
|
||
`spectrum = true` (default) in `cbd-tui.toml` / `cbd.toml` shows the
|
||
bars; `false` hides them. The toggle is a *display* choice — the
|
||
server always computes and streams when something is playing. At
|
||
household scale one small FFT task at 20 fps is not worth gating on a
|
||
per-client preference, and keeping the server unconditional means any
|
||
client (TUI, web) can show bars without a negotiation. The FFT only
|
||
runs while audio is actually flowing (D2), so an idle server is idle.
|
||
|
||
### D4 — rendering
|
||
|
||
The TUI draws the bars in the now-playing pane, directly under the
|
||
progress gauge, as a single row of vertical block glyphs
|
||
(`▁▂▃▄▅▆▇█`) whose heights track the bins, in the accent color. The web
|
||
client renders the same bins as CSS-height bars for parity. Both simply
|
||
consume the latest `SpectrumFrame`; neither computes anything.
|
||
|
||
## Structure
|
||
|
||
```d2
|
||
direction: right
|
||
audio: "audio-player (server)" {
|
||
dec: decoder
|
||
tap: "TappingSource\n→ SpectrumTap ring"
|
||
sink: rodio sink
|
||
dec -> tap -> sink
|
||
}
|
||
fft: "spectrum task\nHann + realfft →\nlog bins, ~20fps"
|
||
stream: "GetUpdateStream\nSpectrumFrame{bins}"
|
||
tui: "TUI now-playing\nblock-glyph bars"
|
||
web: "web client\nCSS bars"
|
||
audio.tap -> fft: snapshot
|
||
fft -> stream
|
||
stream -> tui
|
||
stream -> web
|
||
```
|
||
|
||
## Risks
|
||
|
||
- **Audio-thread cost**: the tap must stay trivial; anything more than
|
||
a store per sample risks underruns. No locks, no allocation, no
|
||
logging on that path.
|
||
- **Torn reads**: the FFT reads the ring while the audio thread writes
|
||
it. Accepted — a visualizer tolerates the occasional stale/mixed
|
||
sample; correctness of playback is never affected (the tap only
|
||
*observes*).
|
||
- **realfft** is pure Rust (no system libs), so it does not complicate
|
||
packaging.
|