Why the Future of Machine Learning Is a Master Algorithm

Algorithms. Is there anything they can’t do?

Yes. Tons of stuff, but they remain at the heart of the internet as we know it. Much of what we are exposed to online courtesy of search engines or Siri or Facebook is surfaced based on algorithms designed to improve performance by gathering information. These are today’s most important — if not most effective — learning machines, but Pedro Domingos, a professor of computer science at the University of Washington, is more concerned with what we’ll be capable of calculating tomorrow.

In his new book, The Master Algorithm, Domingos makes the case that it’s possible we may one day create an algorithm so adept at learning and harnessing information that it will forever change the way we think. That hypothetical algorithm will make Google’s site crawler look like basic arithmetic.

Inverse asked Domingos about his mathematically messianic prophesy and the future of calculation.

Can you give me a brief history of the research and development of machine learning? What are maybe two or three of the biggest milestones people should know for how algorithms have evolved over the last several decades?

Computers got their start around World War II — that’s really when computers science began. From the very beginning, there were people who were writing algorithms, explaining line-by-line what the computer should do. But there were also people, including Alan Turing, who were very interested in this idea of computers learning from experience the way people do.

One of the first milestones was the Perceptron Algorithm: the first neural network. Frank Rosenblatt was the first person to develop it. And it was the beginning of simulation for how the brain learns. It was extremely popular in the ‘50s and ‘60s, but then there was this book called Perceptrons: An Introduction to Computational Geometry that revealed a lot of the limitations. People lost faith in machine learning for about 20 years.

Machine learning came back in the ‘80s when people realized the conventional computer processes didn’t scale. Hand-coding all the knowledge you need to solve problems is too expensive, slow, and brittle.

What would you say is the central thesis of your book?

My thesis is that there is a learning algorithm that can discover any knowledge from data. All the knowledge that human beings have, acquired by experience and evolution, and all the future knowledge that we have yet to acquire like curing cancer — all of this can be learned by an algorithm. There are reasons for and against this idea, and I discuss them in the book. But at the end of the day, we’re only going to find out if I’m right by trying.

There are different paradigms under which different sets of researchers fall under, who have what they call their own master algorithms. The connection is backpropagation, which is really what drives deep learning. Often they are convinced that this is the master algorithm, and that they will solve the whole learning problem with it. I myself don’t think any of those things by themselves are the master algorithm. But we need an algorithm that combines them. Again, the analogy is with the unifying theories you find in physics or biology — like the standard model or central dogma. That’s what we need here, too.

The impact on the world would be revolutionary, in all aspects of life.

What are currently the most important projects or trends in machine learning research that we should be closely following?

As the decades go by, schools of machine learning are ascending or descending. Right now, the connectionists are in descending, while the fastest progress right now is in deep learning. You always read about new research in deep learning, in things like speech recognition or learning from YouTube videos. We’ll see how far that gets — those doing the research think it will get us all the way, while others are skeptical.

A lot of the high-tech companies working in this field are competing to develop both an algorithm that can learn from all the data you produce, as well as the model of you that comes out of that data using those algorithms. It’s an arms race, and we’re going to be seeing more of that. Apple has Siri, Google has Google Now, Microsoft has Cortana, and Facebook has M. What they’re all trying to do is learn about all the data we produce — every last bit of data we put out. We’re gong to see a lot of results from this as we move forward.

Another thing to keep an eye on centers around the symbolist school in machine learning. Those really believe in learning by accumulating knowledge, like by reading text and web. Google’s Knowledge Graph is probably the most famous example of this, but there are a lot of others in most industry and academia, like Tom Mitchell’s NELL (Never Ending Language Learning) project, which continuously learns by reading the web.

What do you envision as the future for machine learning and algorithms in the next several decades?

We’re going to get more and more powerful learning algorithms. This will lead to progress in all of the areas where machine learning is now being used.

But we’re also going to see a change in the nature of computing systems we interact with. Right now, we have these rigidly-programmed things that can be frustrating to use. As they become more based on machine learning, and algorithms replace pure programming, these systems become more flexible and adaptable. Computing will become a more pleasant experience. The world will start adapting to you — you won’t need enter information deliberately.

At the same time, paradoxically, computers will become more invisible. These days, you have to think a lot about how to get your laptop or smartphone to do what you want. Computing will move more into the background as we move into the future.