Realistic 3D audio can greatly enhance the sense of presence in
a virtual environment. We introduce a framework for capturing,
transmitting and rendering of 3D audio in presence of other
bandwidth savvy streams in a 3D Tele-immersion based virtual
environment. This framework presents an efficient implementation
for 3D Binaural Spatialization based on the positions of current
objects in the scene, including animated avatars and on the fly
reconstructed humans. We present a general overview of the
framework, how audio is integrated in the system and how it can
exploit the positions of the objects and room geometry to render
realistic reverberations using head related transfer functions.
The network streaming modules used to achieve lip-synchronization,
high-quality audio frame reception, and accurate localization for
binaural rendering are also presented. We highlight how large
computational and networking challenges can be addressed efficiently.
This represents a first step in adequate networking support for Binaural
3D Audio, useful for telepresence. The subsystem is successfully
integrated with a larger 3D immersive system, with state of art capturing
and rendering modules for visual data.
Mauro DA, Mekuria R, Sanna M. Binaural Spatialization for 3D immersive audio communication in a virtual world. Audio Mostly - a Conference on Interaction with Sound, 18-20 September, 2013, Piteå (Sweden)