Holly+, Holly Herndon's digital AI voice twin

Holly Herndon is an American singer, composer, and AI musician who lives and works in Berlin. She has been working with machine learning for years and uses AI tools in her music production. Stylistically, her music moves between pop, folk, choral music, and electronica.

The album Proto

Together with her partner Mathew Dryhurst and developer Jules LaPlace, Herndon has already used neural networks on her 2019 album Proto. During the production of the album, an AI model was trained with vocal recordings of Herndon and a Berlin vocal ensemble. 

The outputs of the network are then constantly compared with the inputs and adjusted. Through this training, the system learns and becomes more and more accurate.

“Spawning” the AI tool

Neural networks are a form of AI systems modeled on how the human brain works, more specifically how biological nerve cells work. Artificial neural networks are trained by feeding them with data that includes both the input and the resulting output. For the AI tool, Herndon uses the metaphor of an AI baby, a “spawn”. “Spawning” is what she calls the process of feeding data by the human collective.

The goal is to adjust the weights of the connections between the digital neurons so that the outputs of the network are as close as possible to the desired outputs. 

The AI music tool Holly+

With the “Holly+” project, Herndon goes one step further. In collaboration with the company NBHS (Never Before Heard Sounds), Herndon has created a “digital twin” of her voice with Holly+.

On the project website, she provides users with a tool to experiment with Holly+ and produce their music. For this purpose, monophonic or polyphonic audio files can be uploaded and then a “transformed” Holly+ audio file can be downloaded.

© Holly Hendon

Screenshot of Holly+ on holly.plus

The Herndon voice model is a neural network that is trained and, as a result, enables texture mapping. This allows the sound characteristics of a source sound to be transferred to a target sound. The quality of the results depends heavily on how differentiated and detailed the training material is. So she fed the model with as many facets of her speaking and singing voice as possible.

Let’s give it a try. Here are sound samples that I’ve uploaded to the Holly+ website one by one. Once uploaded, the files were processed and I was able to download the three new “transformed” sound files.

A new exploitation model for AI Work

But that’s not all: Herndon has also thought about an exploitation model that does justice to the new possibilities of AI tools. It refers to the principle of “My Collective Ass”, which was proposed by the theorist McKenzie Wark.

Wark mentions three phases in the development of works of art in which rarity and collector’s value differ. The first phase is that of the old masters (e.g. Rembrandt). This is where the work of art detaches itself from the handcraft. Works of art are produced collectively in his studio, he acts as the author. A new era is beginning in industrial capitalism. Goods are now produced by workers in factories. The rarity and collectability of the artwork are determined by the fact that the artist is an original personality. Today, the work of art is no longer a commodity, but a financial instrument, a derivative, according to Wark.

The basic idea that Herndon adopts at Wark is that the more prominently a work of art is seen/heard, the more valuable a certified original becomes.

A DAO as a management model

Now that users have downloaded their Holly+ voice part, they can do whatever they want with it. However, if commercial use is desired, Herndon suggests that a DAO (Decentralised Autonomous Organization), as the administrator of the Holly+ voting model, vote on the certification of new works. The DAO is made up of token holders. These certified or licensed works are created with Holly+ contributions that contribute to increasing the value of the voice. 

The works of the users are intended to reduce the value of the voice, for example by using discriminatory texts or evoking negative associations. DAO token holders are entitled to a portion of the profits from the new works. And that for an unlimited period.

Herndon sees a virtuous cycle set in motion by releasing tools, allowing users to use their digital voice counterparts, and the profits benefiting artists, DAO members, and the further development of the tools.

Artificial intelligence drives us into collectives: although we all contribute to its training datasets, it's impossible to identify which specific inputs influenced the generated outputs. Holly Herndon and Mat Dryhurst guide the audience through strategies that leverage collective action to empower artists and the public, giving them greater control over these emerging technologies.

How AI is changing the music world

In this video, released on Freethink.com in 2023, Mathew Dryhurst and Holly Herndon summarize their vision of the future of AI in the music world in an entertaining way. At the beginning of the video, Dryhurst demonstrates the new, optimized version of Holly+ in real-time.

There is a lot to read about Herndon’s work on the Internet. With her practice-oriented and scientific-artistic background, her involvement with AI tools is an important voice when it comes to discussing the rights of the use of AI art in the here and now. The podcast Interdependence by Holly Herndon and Mat Dryhurst is also recommended.

And here a more recent article: Holly Herndon´s Infinite Art published in The New Yorker.