Artificial General Intelligence with novel Deep Reinforcement Learning approach

Another day, another paper on Artificial Intelligence (AI) on this Blog. Today is also marked by the spell of novelty, innovation and better methodologies. It has been said that there is excessive hype regarding the scope and possibilities of current AI research, with some analysts and pundits saying that we will need to be wary of undesirable outcomes from these efforts and skeptical of our control capacity over AI, while others defending that all of this is just a fad that will eventually get to nothing.

Well, as common wisdom say, normally the truth is in the middle. Today’s paper reminds us exactly that, not discouraging the developments, on the contrary, by bringing a novel approach to what might in the end be of greater importance to AI: sustained improvement towards achieving Artificial General Intelligence and dealing with the still largely inefficient process of data processing by machines in real life settings. I initially got  the knowledge of the paper while I was reading an excellent journalistic piece from British reference daily newspaper The Guardian. In it the reporter speaks with knowledgeable researchers from Imperial College London that were present at the recent REWORK London Deep Learning Summit  on the current developments being achieved with AI, with somewhat similar views, but with slight differences regarding what machine intelligence really can achieve these days; one of the researchers thinking of data as today’s oil of global economy, while other, more prudent and perhaps closer to the truth, with a view of data resembling the XVIIIth Century economic revolution brought by coal and the emergence of a robust coal industry that fueled the Industrial Revolution. This last view may be the correct one.

Current Deep Learning neural networks are great technological accomplishments, but they still have some caveats that render their role in AI still a very slow work in progress. For instance there is an issue about the trade-off between the amount of data required for those systems to perform theirs tasks well versus the quality and efficiency with which the same systems use that data, and that was evident to anyone who read the article from The Guardian. But in the same article we could all find the link to today’s reviewed research paper, that tries to address the issue of Deep Neural Networks not achieving proper general Intelligence with a novel approach  that uses Deep Reinforcement Learning in place of the former approach. And the crucial point has to do with the issue of data efficiency processed by the neural network, where the paradigm of reinforcement learning might be of help, even if it still needs lots of data to perform well at its tasks.

The interesting point of the paper is that it tries to combine former symbolic approaches to AI, that were failures empirically, but which were a better mimic of high-level cognitive processes and more suitable for the kind of human reasoning involved in transparent cognitive verifiability of conceptual knowledge, with the current data intensive approaches. And the results looked promising:

Towards Deep Symbolic Reinforcement Learning

Deep reinforcement learning (DRL) brings the power of deep neural networks to bear on the generic task of trial-and-error learning, and its effectiveness has been convincingly demonstrated on tasks such as Atari video games and the game of Go. However, contemporary DRL systems inherit a number of shortcomings from the current generation of deep learning techniques. For example, they require very large datasets to work effectively, entailing that they are slow to learn even when such datasets are available. Moreover, they lack the ability to reason on an abstract level, which makes it difficult to implement high-level cognitive functions such as transfer learning, analogical reasoning, and hypothesis-based reasoning. Finally, their operation is largely opaque to humans, rendering them unsuitable for domains in which verifiability is important. In this paper, we propose an end-to-end reinforcement learning architecture comprising a neural back end and a symbolic front end with the potential to overcome each of these shortcomings. As proof-of-concept, we present a preliminary implementation of the architecture and apply it to several variants of a simple video game. We show that the resulting system — though just a prototype — learns effectively, and, by acquiring a set of symbolic rules that are easily comprehensible to humans, dramatically outperforms a conventional, fully neural DRL system on a stochastic variant of the game.

As said the resulting system is still just a prototype and the applicability to wider general contexts still needs to be tested. But it really seemed that if AI is ever to achieve the level of Artificial General Intelligence it ambitions, a path along these lines may be warranted some time in the future.

Featured Image: Why data is the new coal – The Guardian


2 thoughts on “Artificial General Intelligence with novel Deep Reinforcement Learning approach

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s