Marine soundscape ecosystems at risk: a data-driven spatial-sound interactive experience to raise awareness


504 Ways Of Listening is an immersive interactive installation that transforms underwater acoustic data into three-dimensional soundscapes. It reveals the impact of human-made noise in the oceans and invites deep listening as a way to reconnect with marine ecosystems.


Pier Alfeo, Alessio Mastrorillo



Veröffentlichung


Reading time
6 Minuten

Listen to article
Loading the Elevenlabs Text to Speech AudioNative Player...

Human impact on Marine Soundscapes

Sound is vital for many marine species, enabling communication, navigation and awareness of their surroundings. Many organisms use acoustic signals to find mates, detect predators and coordinate group behavior. The soundscape of an ecosystem is key to its ecological health.

However, human-made noise — such as from shipping, construction and energy exploration — is increasingly disrupting these natural underwater soundscapes.

The effects of noise pollution are diverse and serious. It can mask important biological sounds, cause stress, disrupt mating calls, impair hunting dynamics and even lead to habitat abandonment. These changes contribute to a decline in biodiversity.

Hence, acoustic monitoring has become essential to detect and understand these disruptions.

Calling for reconnection

Mitigating the impact of anthropogenic noise requires a multifaceted approach: beyond the adoption of quieter technologies, the implementation of acoustic regulations, and the establishment of protected areas, it is essential to act on a cultural level by promoting a renewed culture of listening and sound awareness. 

Our ocean will be determinant in many of the things that happen in the choices that we have ahead, on where we live and how our economy moves, our ecosystems, our energy and how we deal with some of the risks that come from the climate change that we are going to have to adapt to in the future. So, our understanding of the Ocean is central to our understanding of our own prospects as a society and as an economy - John Bell, European Commission--Digital Twin Ocean

With a 2030 target, the EU Mission Restore our Ocean and Waters aims to protect and restore the health of our ocean and waters through research and innovation, citizen engagement, and blue investments. The Mission’s new approach addresses the ocean and waters as a unified system and plays a key role in achieving climate neutrality and nature restoration. In line with this mission, our work is positioned within the dimension of sensory and cultural engagement: a form of active participation that recognizes listening as a political, aesthetic, and epistemological practice - capable of generating shared ecological value.

Acoustic pollution has already profoundly transformed marine and terrestrial ecosystems, as well as our perceptual habits - altering languages, communication, and, in urban contexts, even musical production, as demonstrated by the phenomenon of musical pollution. In this light, the practice of active listening becomes an act of ecological citizenshipThe issue of musical pollution must be assessed from this broader perspective of the relationship between humans and their sound environment - both in terms of the health of individuals and ecosystems, and in terms of the value of the sound environment as a public domain, a common good, an integral part of the ecosystem that must be protected.

To foster awareness around these issues and to promote listening as a form of care, we are developing a deep, blindfolded, and interactive sonic experience - an immersive installation - that uses scientific underwater passive acoustic data and new technologies, placing them at the service of the common good.

504 Ways Of Listening

504 Ways of Listening is an immersive, spatial sound experience that invites critical reflection on how we listen, who designs that listening, and what forms of knowledge become possible through sound. This interactive installation involves a single blindfolded performer at a time, inside an Ambisonics dome and surrounded by an audience of listeners.

Dome 3D sketch

The prototype of 504 Ways of Listening in a sketch, with a dome of 23 speakers and the control platform.

Thanks to the Italian National Research Council (CNR) and its high-fidelity hydrophone recordings, we have access to extensive datasets from otherwise inaccessible marine environments. These recordings provide rare access to underwater acoustic ecologies that are otherwise difficult to perceive, forming the empirical basis for our installation’s audio content and structure. The development unfolds in two phases: one asynchronous, focused on preparing and analyzing the data and one real-time, dedicated to sound synthesis and interactive control detailed in the Methodology section. Interaction with 504 unfolds in a blindfolded state, with the performer’s vision entirelyblinded.

Using a VR headset as a gyroscopic sensor and a custom digital Braille keyboard, the performer moves through a multidimensional acoustic world built from real underwater recordings collected by the Italian National Research Council. These high-fidelity recordings capture the invisible ecologies of the ocean — sounds of marine life, environmental shifts and distant human activities— normally inaccessible to our senses.

Video URL
DDMM test framework

State-of-the-art demonstration of Data Driven Multiverse Manipulation real-time framework

The system, powered by our custom software Data Driven Multiverse Manipulation, transforms nine different audio features—like pitch, energy and spectral contrast—into spatial coordinates. Each permutation of audio features and coordinates becomes a new “universe”, creating 504 distinct ways of organizing and listening to the data. Hence, each performance is unique.

plots

This plot illustrates how the binary encoding system affects segment clustering, using a 2-minute slice of the dataset (see Data origin at chapter 2.2) displaying up to 2 kHz in sonogram for easier interpretation. The displayed universe uses RMS energy for the 𝑥-axis, point size and amount of blue; Onset Strength for the 𝑦-axis; Spectral Bandwidth for the 𝑧-axis; Spectral Centroid for the scalar 𝑤 and the amount of red. Finally, Zero Crossing Rate is used for the amount of green. This combination of features appears effective in clustering sound events composed of impulses, separating them based on the frequency with the greatest energy. In Plot 1, the energy peak is located between 1.7 and 2.1 kHz, while in Plot 2 it lies between 300–500 Hz.

plots

The dataset (see Data origin at chapter 2.2) showing one of the possible Universes, obtained by assigning RMS energy to the 𝑥-axis, to the point size and to the amount of blue (RGB encoding), Onset Strength to the 𝑦-axis, Spectral Bandwidth to the 𝑧-axis and Spectral Density to the scalar 𝑤. Additionally, Spectral Centroid determines the amount of red and Zero Crossing Rate the amount of green. The effects of the quantization required for binary encoding are also visible as bands of points.

Data Driven Multiverse Manipulation (DDMM)

Data Driven Multiverse Manipulation (DDMM) is a software framework, developed by Pier Alfeo and Alessio Mastrorillo, structured in two main phases: an offline phase, primarily dedicated to the analysis of audio recordings (dataset), and a real-time phase focused on managing the Ambisonics sphere and reading the analysis data.

Framework overview

This framework was developed using the programming languages Python and MaxMSP, along with the REAPER software.

The choice of Python as the programming language for the offline operations stems from its strong suitability for dataset analysis, including audio data, thanks to its extensive ecosystem of specialized libraries, such as NumPy, Librosa, Pydub, and SciPy, which we employed. Additionally, Python features a simple and readable syntax and offers widely adopted tools for Machine Learning and Artificial Intelligence, which are among the future developments planned for DDMM.

MaxMSP proves to be the optimal choice for real-time operations, as it is a powerful environment for creating interactive applications in the audio field, from music to multimedia arts. It features a visual programming environment that is immediate and modifiable in real time, valued for its flexibility in processing audio and MIDI signals and in the interactive control of sound, video, and physical devices.

Its modular and performance-oriented nature makes it both a creative and technical tool. Moreover, it includes powerful libraries for handling complex multichannel techniques, such as Ambisonics via Spat~ by IRCAM, and it offers the ability to write sample-level optimized algorithms using Gen~, a MaxMSP subsystem that compiles code to reduce latency and improve efficiency.

Within the digital audio workstation REAPER, the decoding phase of the Ambisonics signals generated with Spat~ is managed using the IEM audio plugin suite. This phase is efficiently handled by REAPER, which is particularly lightweight, versatile, and well-suited for flexible multichannel signal routing.

Guided by the pleasure of discovery, we have followed an empirical approach that combines intuition and experience from both musical and technical domains. This approach leads us to explore an immensely broad field spanning biology, data science and media art, one with political, ecological and epistemological implications.

We envision a future in which tools like DDMM can be made accessible beyond specialist contexts, contributing to new ways of listening to and understanding the complexity of the living world.

Final Words

504 isn’t just about sound — it’s about listening as a form of connection. This work turns scientific data into an emotional, cultural, and sensory experience. It asks: can we listen our way into the complexity of marine life? Can deep, embodied listening bring us closer to what we can’t see—but urgently need to feel?

By re-imagining environmental and biological recordings as cultural material, 504 invites reflection on the sonic richness of marine life. It asks whether art—and embodied listening—can help us access complex systems that lie beyond our rational comprehension.


Link to Paper: https://zenodo.org/records/15842120




Pier Alfeo & Alessio Mastrorillo

Alessio Mastrorillo, born in 1987 in Italy, is a composer whose works have garnered recognition from prestigious events such as ICMC, SMC, eviMus and Tempo Reale. His musical journey began with a foundation in piano, drums and live music engineering, gradually evolving into the realm of computer music. Fueled by a passion for new technologies, he has actively contributed to international conferences like NYCEMF, CIM, SMC and AIA, showcasing his dedication to pushing the boundaries of music through cutting-edge methodologies. His artistic activity extends to solo and ensemble Live Electronics concerts, where he captivates audiences with a seamless fusion of traditional and electronic elements. He has collaborated with notable groups such as Mediterraneo Radicale and the Tempo Reale Electroacoustic Ensemble, further solidifying his impact on the contemporary music scene. In addition to his artistic career, Alessio teaches Computer Music at the Conservatory of Ravenna and Modena, sharing his expertise with the next generation of musicians and researchers.

Pier Alfeo (born 1985) is a sound and transmedia artist working across sound sculpture, installations, electronic music, and immersive audiovisual projects. He has exhibited and performed at international festivals in Italy, France, UK, Germany, Switzerland, the Netherlands, Cyprus, Serbia, Ukraine, Czech Republic, and Kenya, alongside artists such as Cod.Act, Quayola, Robert Henke, Fennesz, and Murcof. In 2019, he held his first solo show Incisione su Silenzio at Doppelgaenger Gallery (Bari) and won the Italian National Art Prize (Electronic Art). He participated in the Biennale Tecnologia (Turin, 2020), receiving the “Dynamic Display” award. In 2023, he was a finalist for the Re:Humanism 3 Art Prize and took part in a residency at S.O.D.A. in Manchester. In 2024, he joined RE-FEST by CultureHub and PhEST Festival. In 2025, he presented 504 Ways of Listening and the related publication at the SMC Conference in Graz, in collaboration with Italy’s National Research Council.




Original language: English
Article translations are machine translated and proofread.