You might have heard the AI fake of Joe Biden telling New Hampshire voters to stay home. Or about the Zoom scammer using a fake video of an executive to defraud a Hong Kong company of $25 million. These are deepfakes: AI-generated video or audio, made to mimic the appearance or voice of a real person.

Advances in AI are helping dedicated scammers train more and more realistic voice models. Creating many deepfake voices is a labor-intensive process, but AI can help with that too. We had a hunch that an AI system could do most of what a scammer does entirely on its own. So we built Ursula. If you give Ursula a person’s name, it will search the web for video or podcasts including them, extract the portions of the audio that contain their voice, and train a deepfake voice from those clips.

To test it out, we gave Ursula the names of 100 trusted media personalities. In about 30 minutes, we had 80 of their voices. Check out the results:

But people have been scamming each other for millennia, right? Yes. But in the past, if you were the target of a highly personalized and convincing scam, that meant that someone had spent a lot of time and effort targeting you specifically. Because it was expensive, it was also relatively rare.

AI systems are changing this: Ursula took one engineer less than two weeks to build, mostly using AI systems that were released over the last two years. With the tool, it only costs $1 to steal a new voice. It will soon get even cheaper for scammers to mount shockingly personal attacks against unsuspecting people. You can hear an example of what that might sound like in this video from Control AI:

What does this mean?

The most obvious implication of this demonstration is that you can no longer safely assume that the person calling you on the phone, or over Zoom, is who they seem to be.

But these same AI systems will enable all kinds of scalable deception as well: not only widespread fraud, but also political manipulation and misinformation via social media. The same technology that we used to automatically research targets of our voice cloning system can easily be used to research and deceive people on Facebook.

Companies are working to mitigate some of the risks: Google has announced automated real-time scam detection for phone calls, and an industry coalition is working on standards for verifiable video. But technologies like this will take time to reach maturity, and even more time to reach widespread use. It’s important to be prepared in the meantime.

How can I protect myself and the people I care about?

You can protect yourself from phishing by following two steps:

First, Notice when the stakes are high: If someone asks you for something over the phone or the internet, that’s when you need to be suspicious — even if they seem to be someone you know and trust! Here are some more specific warning signs:

  1. They’re contacting you from a number or account you don’t recognize.
  2. There’s a sense of pressure or urgency. Maybe they say they’re in jail, or stuck at an airport with no phone or money.
  3. They’re asking you for information you wouldn’t share publicly (passwords, social security numbers, bank account numbers).
  4. They’re asking you to go to a specific website, click a specific link, or call a specific number.
  5. They’re asking you to send them money.

These signs don’t always mean you’re being phished, but they are common features of scams. If you notice any of the above, remember to do step 2:

Second, Verify identities: When the stakes are high, always double check the identity of the person you think you’re talking to! The best way to do this is an “out-of-band” check: contact them using a second communication method. If they called you on the phone, check with them on Facebook or WhatsApp. If they messaged you over Facebook, send them a text. If they say they can’t — for example, because they’re calling from a jailhouse phone — verify their identity by asking them questions about your shared history. If it’s your bank calling, hang up and call them back at a number on their public website.

It’s not possible to be totally safe from scammers. But following these two steps will protect you from the most common and dangerous scams.