CyberGrape: A Tactile Spatial Audio System

CyberGrape was my Senior Capstone project, a yearlong project that all CS Engineers embark on at Tufts. Our particular project was to create a tactile spatial audio system. It's an arduous task to mix multiple audio files into one surround sound file, where it sounds like each audio stream is moving around your head in 3D space. Our project makes the process easier by providing a set of "source blocks" which you can move around a "listener block" to record positional data. Then, a computer can use the positional data to mix multiple sound streams together into one surround sound audio file.

The system uses two Bluetooth 5.1 XPLR angle-of-arrival antenna boards and their sister tags. The tags, which emit Bluetooth signals, were placed in the source blocks. Two antenna boards were placed in the listener block to encode a 360-degree range of positional data capture, managed by a Raspberry Pi Pico W. The angle-of-arrival data is transmitted from the listener block to a central computer, at which point it can be interpreted by a Rust software pipeline. Depending on user-provided instruction, the software pipeline either writes the recorded positional data to our serialization format, or uses it to mix provided audio files into a single two-channel binaural audio file.

Find the final report here, the host computer software here, and the embedded software for the Raspberry Pi Pico here.