max planck institut
MPII Max Planck Society

RMM: Reinforced Memory Management for Class-Incremental Learning

Yaoyao Liu1   Bernt Schiele1   Qianru Sun2

1Max Planck Institute for Informatics   2Singapore Management University  
Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS), 2021


Class-Incremental Learning (CIL) trains classifiers under a strict memory budget: in each incremental phase, learning is done for new data, most of which is abandoned to free space for the next phase. The preserved data are exemplars used for replaying. However, existing methods use a static and ad hoc strategy for memory allocation, which is often sub-optimal. In this work, we propose a dynamic memory management strategy that is optimized for the incremental phases and different object classes. We call our method reinforced memory management (RMM), leveraging reinforcement learning. RMM training is not naturally compatible with CIL as the past, and future data are strictly non-accessible during the incremental phases. We solve this by training the policy function of RMM on pseudo CIL tasks, e.g., the tasks built on the data of the zeroth phase, and then applying it to target tasks. RMM propagates two levels of actions: Level-1 determines how to split the memory between old and new classes, and Level-2 allocates memory for each specific class. In essence, it is an optimizable and general method for memory management that can be used in any replaying-based CIL method. For evaluation, we plug RMM into two top-performing baselines (LUCIR+AANets and POD+AANets) and conduct experiments on three benchmarks (CIFAR-100, ImageNet-Subset, and ImageNet-Full). Our results show clear improvements, e.g., boosting POD+AANets by 3.6%, 4.4%, and 1.9% in the 25-Phase settings of the above benchmarks, respectively.

Fig 1.  (a) Existing CIL methods allocate memory between old and new classes in an arbitrary and frozen way, causing the data imbalance between old and new classes and exacerbating the catastrophic forgetting of old knowledge in the learned model. (b) Our proposed method -- Reinforced Memory Management (RMM) -- is able to learn the optimal and class-specific memory sizes in different incremental phases. Please note we use orange, blue and green dots to denote the samples observed in the (i-1)-th, i-th, and (i+1)-th phases, respectively


Please cite our paper if it is helpful to your work:

  author       = {Liu, Yaoyao and
                  Schiele, Bernt and
                  Sun, Qianru},
  title        = {RMM: Reinforced Memory Management for Class-Incremental Learning,
  booktitle    = {Advances in Neural Information Processing Systems 35: Annual Conference
                  on Neural Information Processing Systems 2021, NeurIPS 2021},
  year         = {2021}

Copyright © 2021 Max Planck Institute for Informatics | Imprint / Impressum | Data Protection / Datenschutzhinweis