Autonomous cross-domain knowledge transfer in lifelong policy gradient reinforcement learning

Haitham Bou Ammar, Eric Eaton, José Marcio Luna, Paul Ruvolo

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

46 Scopus citations

Abstract

Online multi-task learning is an important capability for lifelong learning agents, enabling them to acquire models for diverse tasks over time and rapidly learn new tasks by building upon prior experience. However, recent progress toward lifelong reinforcement learning (RL) has been limited to learning from within a single task domain. For truly versatile lifelong learning, the agent must be able to autonomously transfer knowledge between different task domains. A few methods for cross-domain transfer have been developed, but these methods are computationally inefficient for scenarios where the agent must learn tasks consecutively. In this paper, we develop the first cross-domain lifelong RL framework. Our approach efficiently optimizes a shared repository of transferable knowledge and learns projection matrices that specialize that knowledge to different task domains. We provide rigorous theoretical guarantees on the stability of this approach, and empirically evaluate its performance on diverse dynamical systems. Our results show that the proposed method can learn effectively from interleaved task domains and rapidly acquire high performance in new domains.

Original languageEnglish
Title of host publicationIJCAI 2015 - Proceedings of the 24th International Joint Conference on Artificial Intelligence
EditorsMichael Wooldridge, Qiang Yang
PublisherInternational Joint Conferences on Artificial Intelligence
Pages3345-3351
Number of pages7
ISBN (Electronic)9781577357384
StatePublished - 2015
Event24th International Joint Conference on Artificial Intelligence, IJCAI 2015 - Buenos Aires, Argentina
Duration: Jul 25 2015Jul 31 2015

Publication series

NameIJCAI International Joint Conference on Artificial Intelligence
Volume2015-January
ISSN (Print)1045-0823

Conference

Conference24th International Joint Conference on Artificial Intelligence, IJCAI 2015
Country/TerritoryArgentina
CityBuenos Aires
Period07/25/1507/31/15

Fingerprint

Dive into the research topics of 'Autonomous cross-domain knowledge transfer in lifelong policy gradient reinforcement learning'. Together they form a unique fingerprint.

Cite this