|

Sound-reactive environment

Revisiting the sound analysis visualization to explore ideas for a sound-reactive environment...

OKLCH gradient

open full size ↗

This is a visualization of OKLCH, one of the best perceptually accurate and computationally performant color models. Hue runs 0° to 360° left to right; lightness runs 0 to 1 bottom to top. At each cell the chroma is set to the largest value that keeps the color inside the sRGB gamut.

2D sound-reactive color mapping

open full size ↗

Seven fill modes, each mapping sound data to OKLCH coordinates across the screen. Hue, lightness, and chroma are computed from FFT magnitudes, waveform phase, and spectral history — the gamut boundary determines maximum chroma at every point automatically.

Mode index
Anisotropic wave

Twelve cosine plane waves at equal angular intervals, with the \(x\)-axis compressed by \(0.4\times\) before computing dot products. The compression concentrates fringes horizontally and stretches them vertically — the same mechanism as a diffraction grating producing elongated spectra. Hue is assigned by \(H = x \cdot 0.8 + I\pi\), so a left-to-right color sweep is modulated by the interference value at each pixel.

Normal phase portrait

32 waveform samples are plotted as \((w(t),\, w(t + \tau))\) where \(\tau\) cycles through four lag values \(\pm 0.07\) and \(\pm 0.18\). A Gaussian density kernel accumulates over all points, so regions the signal visits often are brighter. Sustained tones trace closed ellipses; percussive transients produce short jagged bursts. Color shifts from cool to warm along the time axis of the trajectory.

Drifting caustic

Seven heptagonal waves each receive an independent phase offset \(\phi_k(t) = (k+1) \cdot 0.73 \cdot t\) rather than a shared time term. Because the drift rates are incommensurable, the pattern never exactly repeats — it samples an orbit on a seven-torus. The interference sum is passed through \(\frac{1}{2} + \frac{1}{2}\cos(I \cdot 3\pi)\), re-introducing three bands of high-contrast structure within the slowly morphing field.

Spectrogram

The 256 FFT bins are arranged left to right (bass to treble); 256 frames of history scroll upward at 60 fps, covering roughly 4 seconds. Hue maps from warm red at the bass end to cool cyan at the treble end. Amplitude sets both lightness and chroma, so silence is black and a full-amplitude bin reaches maximum gamut chroma.

Bilateral spectrogram

The spectrogram is mirrored left and right about the centre column. Vertical position encodes frequency — bass at the top and bottom, treble near the vertical middle — and the top and bottom halves carry the L and R channels respectively. Distance from the centre column encodes time: the centre holds the current frame, and history fans out toward the left and right edges. Hue rotates through a full cycle every ten seconds.

Spectral hue

128 FFT bins appear as vertical frequency bands, mirrored symmetrically left and right. Bass concentrates at the centre; treble fans out to the left and right edges. Each band's hue encodes its amplitude — cool blue when quiet, warm orange when loud.

Spectral hue-lightness

Like mode 5, but lightness also tracks amplitude — quiet bands are dim and cool-tinted, loud bands are bright and warm. Frequencies that surge in the mix glow visibly against the darker surrounding bands.

Environmental mapping

open full size ↗

The 2D color fill is rendered to a texture and wrapped around the scene as an equirectangular environment map. Each surface point samples the map at its normal direction, so the color pattern painted by the audio becomes the light source for the 3D shape.