max planck institut
MPII Max Planck Society

Mnemonics Training: Multi-Class Incremental Learning without Forgetting

Yaoyao Liu1   Yuting Su2   An-An Liu2   Bernt Schiele1   Qianru Sun3

1Max Planck Institute for Informatics   2Tianjin University   3Singapore Management University  
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition


Multi-Class Incremental Learning (MCIL) aims to learn new concepts by incrementally updating a model trained on previous concepts. However, there is an inherent trade-off to effectively learning new concepts without catastrophic forgetting of previous ones. To alleviate this issue, it has been proposed to keep around a few examples of the previous concepts but the effectiveness of this approach heavily depends on the representativeness of these examples. This paper proposes a novel and automatic framework we call mnemonics, where we parameterize exemplars and make them optimizable in an end-to-end manner. We train the framework through bilevel optimizations, i.e., model-level and exemplar-level. We conduct extensive experiments on three MCIL benchmarks, CIFAR-100, ImageNet-Subset and ImageNet, and show that using mnemonics exemplars can surpass the state-of-the-art by a large margin. Interestingly and quite intriguingly, the mnemonics exemplars tend to be on the boundaries between classes.



The computing flow of the proposed mnemonics training. It is a global BOP that alternates the learning of mnemonics exemplars (we call exemplar-level optimization) and MCIL models (model-level optimization).



Please cite our paper if it is helpful to your work:

author    = {Liu, Yaoyao and Su, Yuting and Liu, An{-}An and Schiele, Bernt and Sun, Qianru},
title     = {Mnemonics Training: Multi-Class Incremental Learning without Forgetting},
booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
pages     = {12245--12254},
year      = {2020}


[1] Li, Zhizhong, and Derek Hoiem. "Learning without forgetting." TPAMI 2017.
[2] Rebuffi, Sylvestre-Alvise, et al. "icarl: Incremental classifier and representation learning." CVPR 2017.
[3] Wu, Yue, et al. "Large scale incremental learning." CVPR 2019.
[4] Hou, Saihui, et al. "Learning a unified classifier incrementally via rebalancing." CVPR 2019.

Copyright © 2020 Max Planck Institute for Informatics | Imprint / Impressum | Data Protection / Datenschutzhinweis