The Dangerous Convergence: AI-Enabled Voice Cloning and Deepfaked Kidnapping

Voice Cloning Enabled by Artificial Intelligence Poses Growing Danger

The Incident and the Rise of Deepfake Scams

An incident earlier this year highlighted the growing danger of voice cloning enabled by artificial intelligence (AI). In this case, a cybercriminal attempted to extort $1 million from a woman in Arizona by claiming to have kidnapped her daughter. The criminal used deepfake technology to create a voice recording that sounded identical to the daughter, making the worried mother believe her child was in danger. This incident is just one example of a rapidly increasing trend where cybercriminals exploit AI-enabled tools to manipulate and deceive people.

In response to the rising threat, the FBI issued a warning in June, cautioning consumers about criminals using AI to manipulate videos and photos for extortion attempts. This misuse of deepfake technology has added a new twist to imposter scams, which cost US consumers $2.6 billion in losses last year, according to the Federal Trade Commission.

The Ease of Voice Cloning and the Role of Social Media

Creating deepfake videos and audio, like the one used in the kidnapping scam, doesn’t require much biometric content. Even a few seconds of audio from an individual’s social media posts can be enough for threat actors to clone their voice. Several AI-enabled voice cloning tools are readily available, such as ElevenLabs’ VoiceLab, Resemble.AI, Speechify, and VoiceCopy.

With the cost of these tools often being less than $50 a month, they are easily accessible to those involved in imposter scams. Additionally, the dark web provides a plethora of videos, audio clips, and other identity-containing data that threat actors use to identify targets for their scams. They can use AI tools like ChatGPT to manipulate data from various sources, including video, voice, and geolocation, to narrow down potential victims for voice cloning and other scams.

Implication for Cybersecurity and Ethics

The rise of deepfake scams poses significant challenges for cybersecurity professionals. Malicious individuals can make use of voice cloning to deceive victims and extract ransom amounts. There is also a risk of threat actors blending voice and video in ransomware attacks and cyber-extortion schemes.

As AI technology rapidly advances, it is crucial to address the ethical implications and potential misuse of these tools. Voice cloning vendors, such as ElevenLabs, Microsoft, and Meta (Facebook parent company), are aware of the risks and are taking measures to mitigate them. Suggestions include additional account checks, ID verification, copyright verification, and manual approval for voice cloning requests.

Protecting Against Deepfake Scams

It is essential for individuals to be aware of the threat and take precautions to protect themselves from deepfake scams. Here are some recommendations:

• Exercise caution when sharing personal information, photos, and videos online, especially on platforms that may not have robust privacy measures.
• Regularly review privacy settings on social media platforms and limit the visibility of personal content.
• Be wary of any unexpected requests for money, particularly if they involve a seemingly distressed loved one.
• Do not rely solely on voice identification in high-stakes situations. Whenever possible, use additional verification measures to confirm the identity of individuals.
• Stay informed about the latest cybersecurity threats and scams to better protect yourself and your loved ones.

Voice cloning technology has the potential for both beneficial and malicious use. As AI continues to evolve, it is crucial to strike a balance between innovation and protecting individuals from its harmful consequences. The responsibility lies not only with vendors and technology developers but also with individuals to be vigilant and proactive in safeguarding their digital security.