Reducing Catastrophic Forgetting With Learning on Synthetic Data

TytułReducing Catastrophic Forgetting With Learning on Synthetic Data
Publication TypeConference Paper
Rok publikacji2020
AutorzyMasarczyk W, Tautkute I
Conference NameProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops
Date PublishedJune
Abstract

Catastrophic forgetting is a problem caused by neural networks' inability to learn data in sequence. After learning two tasks in sequence, performance on the first one drops significantly. This is a serious disadvantage that prevents many deep learning applications to real-life problems where not all object classes are known beforehand; or change in data requires adjustments to the model. To reduce this problem we investigate the use of synthetic data, namely we answer a question: Is it possible to generate such data synthetically which learned in sequence does not result in catastrophic forgetting? We propose a method to generate such data in two-step optimisation process via meta-gradients. Our experimental results on Split-MNIST dataset show that training a model on such synthetic data in sequence does not result in catastrophic forgetting. We also show that our method of generating data is robust to different learning scenarios.

URLhttps://openaccess.thecvf.com/content_CVPRW_2020/html/w15/Masarczyk_Reducing_Catastrophic_Forgetting_With_Learning_on_Synthetic_Data_CVPRW_2020_paper.html

Plik PDF: