Always stronger ! ElevenLabs has developed an AI capable of perfectly reproducing human voices. But it didn’t take long before she was hijacked to make racist and homophobic remarks to celebrities…
It’s a fact, technological advances lead to as many abuses as benefits. Just look at the enthusiasm around ChatGPT since its release and the many opportunities that the tool opens up… including for hackers. Indeed, malicious clone applications are multiplying on the App Store and Play Store, while hackers try to use AI to find vulnerabilities in websites and develop malware. In the same vein, the emergence of deepfakes has been accompanied by new revenge porn – by placing someone’s face in a pornographic photo or video – and the dissemination of fake news – such as the false capitulation of the Ukrainian president for example. And it’s not about to stop! Last week, the start-up ElevenLabs released for free to the public the beta version of its new AI, which is able to create voices from a simple written description. Better – or worse – yet, it can take the voice of an existing person, who can then be made to say whatever you want. A false good idea ?
ElevenLabs: diverted recordings with violent remarks
ElevenLabs’ tool is based on GPT-3, the language model that was used for ChatGPT and the DALL-E image generator. Like the latter, it is very simple to use: you just have to register and write a text – only in English for the moment – so that it is then recited by synthetic but realistic voices. It can go a step further by generating a clone of an existing person’s voice from a recording sample that is over a minute long using the VoiceLab feature. In principle, it is a question of using AI for creation – as for the dubbing of a video game for example -, the production of audio books, or even the dubbing of articles or newsletters – which offers therefore greater accessibility to information.
Unfortunately, it didn’t take more than a few days for things to get out of hand. “While we see our technology being massively used in a positive way, we are also seeing an increasing number of cases of voice cloning being misused.” ElevenLabs quickly posted on Twitter. Indeed, journalists from Motherboard discovered on the 4chan forum some audio excerpts in which we hear famous personalities making violent, racist or homophobic remarks. One of them, for example, uses the voice of Emma Watson to read excerpts from Mein Kampf…
Following this, ElevenLabs announced that it was temporarily suspending free access to its platform, while put in place safeguards and boundaries to prevent such misappropriation from occurring again. The AI is currently available again, but the company will set up a tool by next week that will identify the voices from the program. Then, since most of the problematic samples were produced from anonymous and free accounts, it will introduce additional identity verification as soon as possible by making VoiceLab accessible only to paid accounts – in order to at least have their contact details. banks to make them less anonymous. This subscription should cost $5 per month with a free trial to generate 30,000 characters. The free offer with 10,000 characters per month and preformed voices will still be available – but you will also have to fill in your payment information.