REACTO – Web Audio-Visual Experiments

REACTO is a web-based audio visualization suite that transforms uploaded audio or microphone input into 10 distinct real-time 3D experiments. Each visualization maps Web Audio API FFT data to Three.js geometry differently — from a circular spectrum analyzer to a physics-inspired cymatics simulation that implements actual Chladni standing-wave equations. The project was built as both a creative tool and a technical demonstration of audio-reactive real-time graphics in the browser.

Audio visualization on the web rarely goes beyond bar charts. The goal was to explore how far real-time audio analysis could drive 3D geometry — not just height of bars, but topology of terrain, orbits of particles, and resonant modes of vibrating surfaces. Each experiment answers a different question: what does this sound look like when expressed through this particular physical or mathematical metaphor?

Build a multi-visualization audio suite that: (1) shares one AnalyserNode across all experiments; (2) implements fundamentally different mapping strategies per visualization; (3) includes a physically-grounded simulation (cymatics); (4) supports 12 color schemes applicable to any visualization; (5) runs smoothly in fullscreen on mid-range hardware.

• ◈Solo developer — audio engineering, shader authorship, visualization design, frontend

1. 01Cymatics via spectral peak detection and modal equations — the Cymatics visualization detects the top 3 frequency peaks in the FFT and maps them to integer modal indices (m, n, k, l). It then computes sin(m×π×x) × sin(n×π×y) for rectangular modes and sin(k×π×r) × cos(l×θ) for polar modes, blending both. This produces mathematically correct Chladni patterns that actually change with the music's dominant frequencies.
2. 02BufferAttribute direct mutation for geometry updates — all visualizations update Three.js geometry by calling setX()/setY()/setZ() on existing BufferAttributes and setting needsUpdate=true, never creating new geometry. This eliminates GC pressure from per-frame geometry allocation, which would cause frame drops in longer sessions.
3. 03Single AnalyserNode shared via React context — all 10 visualizations read from the same AnalyserNode via a useAudio() hook. Audio setup (createMediaElementSource, connect) happens once in AudioProvider, preventing the Web Audio API graph from growing when switching visualizations.
4. 04Audio band splitting for semantic reactivity — frequency data is divided into bass (bins 0–24), mid (24–96), and high (96–192) ranges. Different visualization layers respond to different bands: terrain elevation responds to bass, particle orbit radius to mid, trail brightness to high. This creates musically coherent animations where visual layers respond to their corresponding sonic registers.
5. 05THREE.MathUtils.lerp for smooth transitions — raw FFT data is noisy and produces jittery visuals. Every value driving geometry (bar height, orbit radius, terrain elevation) is lerped toward the new FFT value with delta-scaled smoothing. This produces fluid motion without introducing perceptible lag.
6. 06Fullscreen-specific camera and lighting adjustments — entering fullscreen increases FOV from 60° to 75° and moves the camera back (Z: 18 → 25). Point light intensity increases by 40% to maintain visual balance at the larger viewport. These are computed deltas, not hardcoded values, so they work at any initial state.

• →Cymatics is the most compelling visualization not because it looks best but because it's grounded in physics — users who know about Chladni patterns recognize the connection and engage differently. Mathematical legitimacy adds a layer of meaning that purely aesthetic choices don't.
• →BufferAttribute mutation requires careful dirty-flagging — forgetting needsUpdate=true causes the GPU to render stale geometry while JavaScript thinks it updated. The bug manifests as frozen geometry that ignores audio input, which looks like an audio connection failure.
• →Band splitting makes reactivity feel musical rather than mechanical — responding to total average frequency produces generic pulsing. Separating bass/mid/high and routing each to different geometric properties creates animations that feel compositionally aware.
• →Shared audio context across visualization switches is non-trivial — naively creating a new MediaElementSource on each switch causes the Web Audio API to throw 'already connected' errors. The AudioProvider pattern (create once, keep alive) solved this but required careful cleanup on component unmount.

Live — deployed to GitHub Pages