Speaking to more than thirty of the nation’s governors on Saturday, Tesla and SpaceX CEO Elon Musk quieted the room with this warning: “AI is a fundamental risk to the existence of human civilization.”

Musk was speaking as part of a panel with Nevada Governor Brian Sandoval at the National Governors Association meeting in Providence, Rhode Island, about a variety of topics, but his comments on artificial intelligence were the most striking amid questions about renewable energy, how to get people to care about NASA, cars without steering wheels, and technology regulation.

“On the artificial intelligence front, I have exposure to the very, most cutting-edge A.I., and I think people should be really concerned about it,” Musk told Sandoval. “I keep sounding the alarm bell, but until people see, like, robots going down the street killing people, they don’t know how to react because it seems so ethereal.”

Musk said “we should be really concerned about A.I.,” and that the emerging technology presents a rare case where government should be proactive in regulation instead of reactive to industry.

“By the time we’re reactive in A.I., regulation’s too late,” he said. “Normally, the way regulation’s set up, a whole bunch of bad things happen, there’s a public outcry, and then after many years, a regulatory agency is set up to regulate that industry. There’s a bunch of opposition from companies who don’t like being told what to do by regulators, and it takes forever. That in the past has been bad but not something which represented a fundamental risk to the existence of civilization.”

Musk said car accidents, airplane crashes, faulty drugs, and bad food are merely harmful to certain parts of the society, but A.I. would affect all of society.

“I’m against overregulation for sure, but I think we better get on that with A.I.,” he said, before noting later that “I really think we need government regulation here.”

Tesla Robot Dance
Automation inside the Tesla factory

“Robots Will Be Able to Do Everything Better Than Us”

Musk said job disruption will be massive when A.I.-powered machines reach their potential, joking “I’m not exactly sure what to do about this,” before adding, “This is really like the scariest problem to me.” He noted the transportation job sector — which accounted for 4.6 million jobs in 2014 — “will be one of the first things to go fully autonomous.”

“The robots will do everything, bar nothing,” he said dryly.

Various technology companies are racing to build better A.I. systems to stay competitive in the market; just look at the work of Facebook with Messenger, or Apple with Siri, and Amazon with Alexa, for some major consumer examples. Around the corner are A.I.-powered autonomous cars and service companies like Uber that’s increasingly waving A.I. into its ride-hailing app.

“Thus we need to the regulators to come in and say, ‘Hey guys, you all really need to pause and make sure this is safe,’” Musk said. “When it’s cool and the regulators are convinced it’s safe to proceed, then you can go, but otherwise, slow down.”

Complicating matters is the fact that the biggest companies developing A.I. right now are publicly held, and stockholders will demand the company aggressively stay ahead of the competition when it comes to developing A.I.

Hickenlooper: What are leaders to do about A.I. regulation?

Colorado Governor John Hickenlooper paraphrased Musk’s claim about A.I. as the biggest “fundamental risk” facing civilization in asking how government leaders should face this challenge — that has remained largely out of the public view — head-on.

“The first order of business would be to try to learn as much as possible, to understand the nature of the issues,” Musk said. “Look closely at the progress that is being made and the remarkable achievements of artificial intelligence.”

As an example, Musk told Hickenlooper about how Google DeepMind defeated the best human Go player and how it can play the top 50 players simultaneously and “crush them all,” calling the rapid progression in a little more than a year “remarkable.”

It got a little more dystopian with this concrete example he gave to Hickenlooper:

“You’re going to see robots that can learn to walk from nothing within hours, like way faster than any biological being,” Musk said.

While killer robots are easy to visualize, the most dangerous threat is a “deep intelligence in the network” that Musk said could start a war by creating fake news, and spoofing email accounts, and sending out fake press releases as a way to manipulate information.

“The pen is mightier than the sword,” he said.


See also: “New Study Confirms That, Yes, Automation Is Taking Jobs”

Photos via Flickr / jurvetson