Towards Deep Symbolic Reinforcement Learning

Marta Garnelo, Kai Arulkumaran, Murray Shanahan
(Submitted on 18 Sep 2016)

 

Abstract:

Deep reinforcement learning (DRL) brings the power of deep neural
networks to bear on the generic task of trial-and-error learning, and
its effectiveness has been convincingly demonstrated on tasks such as
Atari video games and the game of Go. However, contemporary DRL
systems inherit a number of shortcomings from the current generation
of deep learning techniques. For example, they require very large
datasets to work effectively, entailing that they are slow to learn
even when such datasets are available. Moreover, they lack the ability
to reason on an abstract level, which makes it difficult to implement
high-level cognitive functions such as transfer learning, analogical
reasoning, and hypothesis-based reasoning. Finally, their operation is
largely opaque to humans, rendering them unsuitable for domains in
which verifiability is important. In this paper, we propose an
end-to-end reinforcement learning architecture comprising a neural
back end and a symbolic front end with the potential to overcome each
of these shortcomings. As proof-of-concept, we present a preliminary
implementation of the architecture and apply it to several variants of
a simple video game. We show that the resulting system -- though just
a prototype -- learns effectively, and, by acquiring a set of symbolic
rules that are easily comprehensible to humans, dramatically
outperforms a conventional, fully neural DRL system on a stochastic
variant of the game.

Resource Type: