Artificial intelligence was center stage at Google I/O this year, but nothing came as close to science fiction as Google Duplex. This Google Assistant feature is capable of replicating a realistic human voice to make phone calls on your behalf, like a robotic secretary. The demonstration of this soon-to-be-released feature garnered emphatic applause at the tech company developer conference, but others saw Google’s game-changing virtual assistant as a real-life episode of Black Mirror ready to unravel.

Google CEO Sundar Pichai showcased recordings of the A.I. successfully booking a hair appointment and inquiring about the seating situation at a restaurant. In classic fashion, the internet remixed the demo by inserting their own script. The popular channel Funny or Die imagined what it would be like to have Duplex call your parents, and YouTuber Adam Fierman posted a skit of the A.I. carrying out a breakup. Both of these interactions are currently out of the scope of what Duplex can do, but hearing a machine perfectly impersonate a human voice, right down to the umms and ahhs, showed the potential of this technology.

Google Duplex: How It Works — And the Truth Behind the YouTube Parodies

These videos were all made for comedic effect, but they shouldn’t be discounted as impossibilities. Duplex is powered by WaveNet, Google’s machine learning algorithm that is able to train itself on raw audio until it finds its own voice. It could be fed a wide array of voice snippets to learn even more intricate speech patterns and linguistic skills than what we saw at Google I/O.

Google Duplex Security Concerns

While having an A.I. carry out mundane conversations for you might be an introvert’s dream, this technology could also be used for nefarious purposes, many have warned. Impersonating someone over the phone could be used for fraud. Scammers already use social engineering to deceive people, Duplex could make this even easier.

A Google spokeswoman told CNet that the company would be programming Duplex to identify itself as an A.I. whenever it carries out phone calls. This would serve as a way to quell any concerns about how the feature could be used for any criminal activity.

“We understand and value the discussion around Google Duplex — as we’ve said from the beginning, transparency in the technology is important,” she said. “We are designing this feature with disclosure built-in, and we’ll make sure the system is appropriately identified.”

Google might be taking some preemptive steps to make sure Duplex doesn’t get a bad rap before it is released to the public. But it’s hard not to imagine the teens of the future using their Google Assistant to prank-call those of us who still prefer making phone calls ourselves.