The New Hampshire Attorney General's Office announced on Monday that it is investigating possible voter manipulation after complaints of a fake voice that sounded exactly like US President Joe Biden urging voters not to vote in the state's primary.
Researchers also warned of the use of deepfake audio to mimic politicians and leaders, with prominent cases occurring in 2023 in the UK, India, Nigeria, Sudan, Ethiopia and Slovakia.
A person votes in the primary election in New Hampshire, USA. Photo: Getty
Deepfake audio has become a popular form of misinformation due to the emergence of a range of cheap and effective AI tools from startups, according to experts. Microsoft has also developed a new AI model that can copy voices from just three seconds of recording.
“When it comes to visual manipulation, everyone is familiar with Photoshop or at least knows it exists,” said Henry Ajder, an AI and deepfake expert and consultant for Adobe, Meta and EY. “There’s very little awareness of how audio can be manipulated, so to me that really leaves us vulnerable.”
In September, NewsGuard, a system that rates the quality and trustworthiness of news sites, discovered a network of TikTok accounts masquerading as legitimate news sites, using AI-generated content to spread conspiracy theories and political misinformation. Deepfake recordings, including the voice of former US President Barack Obama, garnered hundreds of millions of views.
The fake voiceovers appear to have been generated by a tool provided by Andreessen Horowitz-backed ElevenLabs, while the clips have garnered hundreds of millions of views, NewsGuard said.
“Over 99% of users on our platform are creating interesting, creative, useful content, but we recognize that there are cases of misuse and we have continuously developed and introduced protections to limit them,” ElevenLabs said.
ElevenLabs, founded two years ago by former Google and Palantir employees Piotr Dabkowski and Mati Staniszewski, offers rudimentary AI audio generation tools for free with just a click. Subscription plans range from $1 a month to $330 a month and beyond for those looking for more complex services.
Last year, US intelligence agencies warned in a report that “personalized AI scams have increased significantly due to the release of sophisticated and well-trained AI voice cloning models.”
In addition to financially motivated scams, political experts are now sounding the alarm about viral deepfake audio clips, as well as the use of deepfakes for robocalls or campaigns. “You can build a powerful, widespread disinformation campaign by targeting phones,” said AJ Nash, vice president and distinguished intelligence fellow at cybersecurity group ZeroFox.
Some of these companies have been proactive in finding other ways to combat misinformation. Microsoft has issued an ethics statement, urging users to report any misuse of its AI audio tool. ElevenLabs has built its own detection tools to identify audio recordings created by its system.
During Nigeria's 2023 election, an AI-manipulated clip went viral on social media "purporting to imply an opposition candidate was planning to rig the vote," according to the human rights group Freedom House.
In Slovakia, a fake audio recording of opposition candidate Michal Šimečka apparently plotting to rig the election went viral just days before the country's presidential election in September.
“The New Hampshire deepfake is a reminder of the many ways deepfakes can confuse and cheat,” said Robert Weissman, president of the nonprofit consumer advocacy group Public Citizen. “The political deepfake moment has arrived. Policymakers must move quickly to put safeguards in place or we will face electoral chaos.”
Mai Anh (according to FT)
Source
Comment (0)