Automated Machine Learning – AI software that writes itself

MIT Technology Review posted last 18th of January a nice article about the current developments around Automated Machine Learning.

Automated Machine Learning is a new avenue of research where the developers and researchers try to reach the goal of producing software with the ability to write software by its own. But the so-called automated software aims to write machine learning pipelines and libraries. That is, this aims for Artificial Intelligence becoming more and more autonomous and able to train and test itself.

What is remarkable in the MIT post is the number of research papers it provides, each with its own insight in this endeavour, but with some more advanced or promising than others.

AI Software Learns to Make AI Software

In one experiment, researchers at the Google Brain artificial intelligence research group had software design a machine-learning system to take a test used to benchmark software that processes language. What it came up with surpassed previously published results from software designed by humans.

In recent months several other groups have also reported progress on getting learning software to make learning software. They include researchers at the nonprofit research institute OpenAI (which was cofounded by Elon Musk), MIT, the University of California, Berkeley, and Google’s other artificial intelligence research group, DeepMind.

This is a bit ironic and running against the hype about the shortage of software developers with the required machine learning skill set. But if and when in the future the algorithms will be autonomous and able to run by themselves, where the need and demand for those professionals will be found?

“Currently the way you solve problems is you have expertise and data and computation,” said Dean, at the AI Frontiers conference in Santa Clara, California. “Can we eliminate the need for a lot of machine-learning expertise?”

One set of experiments from Google’s DeepMind group suggests that what researchers are terming “learning to learn” could also help lessen the problem of machine-learning software needing to consume vast amounts of data on a specific task in order to perform it well.

Some prominent researchers in this area, like University of Montreal’s Prof. Yoshua Bengio, welcomed these developments, but cautioned that the deep learning techniques allowing for this to be possible pose significant computational costs. Which points to a future direction of decreasing those costs as an obvious  line of effort. And that will still require human machine learning experts around for the foreseeable future:

The idea of creating software that learns to learn has been around for a while, but previous experiments didn’t produce results that rivaled what humans could come up with. “It’s exciting,” says Yoshua Bengio, a professor at the University of Montreal, who previously explored the idea in the 1990s.

Bengio says the more potent computing power now available, and the advent of a technique called deep learning, which has sparked recent excitement about AI, are what’s making the approach work. But he notes that so far it requires such extreme computing power that it’s not yet practical to think about lightening the load, or partially replacing, machine-learning experts.


On the other hand, the often frustration and difficulty of putting a proper functioning machine learning software pipeline up and running has inspired many researchers to develop autonomous machine learning and deploy it with more or less success. This is an understandable next step for the field, where the business and economic benefits will only materialize when the costs, human and computational, become more attractive to industry and academic institutions. At the same time the data scientist and data engineer will be free to pursue ever more valuable goals and higher-level ideas, compounding the effect of automation further and further.

Otkrist Gupta, a researcher at the MIT Media Lab, believes that will change. He and MIT colleagues plan to open-source the software behind their own experiments, in which learning software designed deep-learning systems that matched human-crafted ones on standard tests for object recognition.

Gupta was inspired to work on the project by frustrating hours spent designing and testing machine-learning models. He thinks companies and researchers are well motivated to find ways to make automated machine learning practical.

“Easing the burden on the data scientist is a big payoff,” he says. “It could make you more productive, make you better models, and make you free to explore higher-level ideas.”


The full list of papers

Following this nice re-post from MIT Technology Review, The Information Age thought it convenient and appropriate to list and present, with the proper titles, all the papers in the post that are developments in automated machine learning. Some of the papers surely deserve a more detailed analysis and review, a task and aspiration this blog have tried before, and will continue to try with improved commitment:

Neural Architecture Search with Reinforcement Learning

Designing Neural Network Architectures using Reinforcement Learning

Learning to reinforcement learn

Learning to Optimize

RLˆ2: Fast Reinforcement Learning via Slow Reinforcement Learning

body text image: Machine Learning for Automated Algorithm Design

featured image: Automated Machine Learning Using Spark Mllib to Improve Customer Experience


2 thoughts on “Automated Machine Learning – AI software that writes itself

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s