Marine soundscape ecosystems at risk: a data-driven spatial-sound interactive experience to raise awareness

Human impact on Marine Soundscapes

Sound is vital for many marine species, enabling communication, navigation and awareness of their surroundings. Many organisms use acoustic signals to find mates, detect predators and coordinate group behavior. The soundscape of an ecosystem is key to its ecological health.

However, human-made noise — such as from shipping, construction and energy exploration — is increasingly disrupting these natural underwater soundscapes.

The effects of noise pollution are diverse and serious. It can mask important biological sounds, cause stress, disrupt mating calls, impair hunting dynamics and even lead to habitat abandonment. These changes contribute to a decline in biodiversity.

Hence, acoustic monitoring has become essential to detect and understand these disruptions.

Calling for reconnection

Mitigating the impact of anthropogenic noise requires a multifaceted approach: beyond the adoption of quieter technologies, the implementation of acoustic regulations, and the establishment of protected areas, it is essential to act on a cultural level by promoting a renewed culture of listening and sound awareness. 

Our ocean will be determinant in many of the things that happen in the choices that we have ahead, on where we live and how our economy moves, our ecosystems, our energy and how we deal with some of the risks that come from the climate change that we are going to have to adapt to in the future. So, our understanding of the Ocean is central to our understanding of our own prospects as a society and as an economy - John Bell, European Commission--Digital Twin Ocean

With a 2030 target, the EU Mission Restore our Ocean and Waters aims to protect and restore the health of our ocean and waters through research and innovation, citizen engagement, and blue investments. The Mission’s new approach addresses the ocean and waters as a unified system and plays a key role in achieving climate neutrality and nature restoration. In line with this mission, our work is positioned within the dimension of sensory and cultural engagement: a form of active participation that recognizes listening as a political, aesthetic, and epistemological practice - capable of generating shared ecological value.

Acoustic pollution has already profoundly transformed marine and terrestrial ecosystems, as well as our perceptual habits - altering languages, communication, and, in urban contexts, even musical production, as demonstrated by the phenomenon of musical pollution. In this light, the practice of active listening becomes an act of ecological citizenship. The issue of musical pollution must be assessed from this broader perspective of the relationship between humans and their sound environment - both in terms of the health of individuals and ecosystems, and in terms of the value of the sound environment as a public domain, a common good, an integral part of the ecosystem that must be protected.

© Courtesy National Research Council of Italy (CNR)

To foster awareness around these issues and to promote listening as a form of care, we are developing a deep, blindfolded, and interactive sonic experience - an immersive installation - that uses scientific underwater passive acoustic data and new technologies, placing them at the service of the common good.

504 Ways Of Listening

504 Ways of Listening is an immersive, spatial sound experience that invites critical reflection on how we listen, who designs that listening, and what forms of knowledge become possible through sound. This interactive installation involves a single blindfolded performer at a time, inside an Ambisonics dome and surrounded by an audience of listeners.

© Alessio Mastrorillo, Pier Alfeo

The prototype of 504 Ways of Listening in a sketch, with a dome of 23 speakers and the control platform.

Thanks to the Italian National Research Council (CNR) and its high-fidelity hydrophone recordings, we have access to extensive datasets from otherwise inaccessible marine environments. These recordings provide rare access to underwater acoustic ecologies that are otherwise difficult to perceive, forming the empirical basis for our installation’s audio content and structure. The development unfolds in two phases: one asynchronous, focused on preparing and analyzing the data and one real-time, dedicated to sound synthesis and interactive control detailed in the Methodology section. Interaction with 504 unfolds in a blindfolded state, with the performer’s vision entirelyblinded.

Using a VR headset as a gyroscopic sensor and a custom digital Braille keyboard, the performer moves through a multidimensional acoustic world built from real underwater recordings collected by the Italian National Research Council. These high-fidelity recordings capture the invisible ecologies of the ocean — sounds of marine life, environmental shifts and distant human activities— normally inaccessible to our senses.

State-of-the-art demonstration of Data Driven Multiverse Manipulation real-time framework

The system, powered by our custom software Data Driven Multiverse Manipulation, transforms nine different audio features—like pitch, energy and spectral contrast—into spatial coordinates. Each permutation of audio features and coordinates becomes a new “universe”, creating 504 distinct ways of organizing and listening to the data. Hence, each performance is unique.

© Alessio Mastrorillo, Pier Alfeo

This plot illustrates how the binary encoding system affects segment clustering, using a 2-minute slice of the dataset (see Data origin at chapter 2.2) displaying up to 2 kHz in sonogram for easier interpretation. The displayed universe uses RMS energy for the 𝑥-axis, point size and amount of blue; Onset Strength for the 𝑦-axis; Spectral Bandwidth for the 𝑧-axis; Spectral Centroid for the scalar 𝑤 and the amount of red. Finally, Zero Crossing Rate is used for the amount of green. This combination of features appears effective in clustering sound events composed of impulses, separating them based on the frequency with the greatest energy. In Plot 1, the energy peak is located between 1.7 and 2.1 kHz, while in Plot 2 it lies between 300–500 Hz.

© Alessio Mastrorillo, Pier Alfeo

The dataset (see Data origin at chapter 2.2) showing one of the possible Universes, obtained by assigning RMS energy to the 𝑥-axis, to the point size and to the amount of blue (RGB encoding), Onset Strength to the 𝑦-axis, Spectral Bandwidth to the 𝑧-axis and Spectral Density to the scalar 𝑤. Additionally, Spectral Centroid determines the amount of red and Zero Crossing Rate the amount of green. The effects of the quantization required for binary encoding are also visible as bands of points.

Data Driven Multiverse Manipulation (DDMM)

504 isn’t just about sound — it’s about listening as a form of connection. This work turns scientific data into an emotional, cultural, and sensory experience. It asks: can we listen our way into the complexity of marine life? Can deep, embodied listening bring us closer to what we can’t see—but urgently need to feel?

By re-imagining environmental and biological recordings as cultural material, 504 invites reflection on the sonic richness of marine life. It asks whether art—and embodied listening—can help us access complex systems that lie beyond our rational comprehension.

Link to Paper: https://zenodo.org/records/15842120