Skip to content

pythonfirst/continual_learning_papers

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

History

308 Commits
reading_group
reading_group
scripts
scripts
Others.md
Others.md
README.md
README.md
bibtex.bib
bibtex.bib

Repository files navigation

Continual Learning Literature

This repository is maintained by Massimo Caccia and Timothée Lesort don't hesitate to send us an email to collaborate or fix some entries ({massimo.p.caccia , t.lesort} at gmail.com). The automation script of this repo is adapted from Automatic_Awesome_Bibliography.

For contributing to the repository please follow the process here

Outline

Classics

Argues knowledge transfer is essential if robots are to learn control with moderate learning times 
Introduces CL and reveals the catastrophic forgetting problem

Surveys

introduces a super simple methods that outperforms almost all methods in all of the CL benchmarks. We need new better benchamrks 
Extensive empirical study of CL methods (in the multi-head setting) 
An extensive review of CL 
An extensive review of CL methods in three different scenarios (task-, domain-, and class-incremental learning)

Influentials

More efficient GEM; Introduces online continual learning 
Proposes desideratas and reexamines the evaluation protocol 
Proposes a reference architecture for a continual learning system 
A model that alliviates CF via constrained optimization 
Introduces generative replay 
Investigates CF in neural networks

New Settings or Metrics

proposes a new continual few-shot setting where spacial and temporal context can be leveraged to and unseen classes need to be predicted 
(title is a good enough summary) 
Proposes a new approach to CL evaluation more aligned with real-life applications, bringing CL closer to Online Learning and Open-World learning 
method for compositional continual learning of sequence-to-sequence models

Regularization Methods

continual learning for non-stationary data using Bayesian neural networks and memory-based online variational Bayes 
Improved results and interpretation of VCL. 
Introduces VCL with uncertainty measured for neurons instead of weights. 
functional regularisation for Continual Learning: avoids forgetting a previous task by constructing and memorising an approximate posterior belief over the underlying task-specific function 
Introduces an optimizer for CL that relies on closed form updates of mu and sigma of BNN; introduce label trick for class learning (single-head) 
Conceptor-Aided Backprop (CAB): gradients are shielded by conceptors against degradation of previously learned tasks 
Formalizes the shortcomings of multi-head evaluation, as well as the importance of replay in single-head setup. Presenting an improved version of EWC. 
A new P\&C architecture; online EWC for keeping the knowledge about the previous task, knowledge for keeping the knowledge about the current task (Multi-head setting, RL) 
Improves on VCL 
Importance of parameter measured based on their contribution to change in the learned prediction function 
Synaptic Intelligence (SI). Importance of parameter measured based on their contribution to change in the loss.

Distillation Methods

Introducing global distillation loss and balanced finetuning; leveraging unlabeled data in the open world setting (Single-head setting) 
Introducing bias parameters to the last fully connected layer to resolve the data imbalance issue (Single-head setting) 
Introducing an expert of the current task in the knowledge distillation method (Multi-head setting) 
Finetuning the last fully connected layer with a balanced dataset to resolve the data imbalance issue (Single-head setting) 
Functional regularization through distillation (keeping the output of the updated network on the new data close to the output of the old network on the new data) 
Binary cross-entropy loss for representation learning & exemplar memory (or coreset) for replay (Single-head setting)

Rehearsal Methods

More efficient GEM; Introduces online continual learning 
projecting the gradients from new tasks onto a subspace in which the neural network output on previous task does not change and the projected gradient is still in a useful direction for learning the new task 
sample selection as a constraint reduction problem based on the constrained optimization view of continual learning 
Controlled sampling of memories for replay to automatically rehearse on tasks currently undergoing the most forgetting 
Uses stacks of VQ-VAE modules to progressively compress the data stream, enabling better rehearsal 
A model that alliviates CF via constrained optimization 
Binary cross-entropy loss for representation learning & exemplar memory (or coreset) for replay (Single-head setting)

Generative Replay Methods

introdudes Dynamic generative memory (DGM) which relies on conditional generative adversarial networks with learnable connection plasticity realized with neural masking 
Extensive evaluation of CL methods for generative modeling 
Extensive evaluation of generative replay methods 
smarter Generative Replay 
Introduces generative replay

Dynamic Architectures or Routing Methods

Proposes a random path selection algorithm, called RPSnet, that progressively chooses optimal paths for the new tasks while encouraging parameter sharing and reuse 
Proposes a reference architecture for a continual learning system
  • Progressive Neural Networks , (2016) by {Rusu}, A.~A.,{Rabinowitz}, N.~C.,{Desjardins}, G.,{Soyer}, H.,{Kirkpatrick}, J.,{Kavukcuoglu}, K.,{Pascanu}, R. and{Hadsell}, R.[bib]
Each task have a specific model connected to the previous ones

Hybrid Methods

Learning task-conditioned hypernetworks for continual learning as well as task embeddings; hypernetwors offers good model compression. 
Approach leverages the principles of deep model compression, critical weights selection, and progressive networks expansion. All enforced in an iterative manner

Continual Few-Shot Learning

proposes a new continual few-shot setting where spacial and temporal context can be leveraged to and unseen classes need to be predicted 
(title is a good enough summary) 
Proposes a new approach to CL evaluation more aligned with real-life applications, bringing CL closer to Online Learning and Open-World learning 
defines Online Meta-learning; propsoses Follow the Meta Leader (FTML) (~ Online MAML) 
Meta-learns a tasks structure; continual adaptation via non-parametric prior 
Formulates an online learning procedure that uses SGD to update model parameters, and an EM with a Chinese restaurant process prior to develop and maintain a mixture of models to handle non-stationary task distribution 
Introduces What & How framework; enables Task Agnostic CL with meta learned task inference

Meta-Continual Learning

Proposes an online replay-based meta-continual learning algorithm with learning-rate modulation to mitigate catastrophic forgetting 
Follow-up of OML. Meta-learns an activation-gating function instead. 
Introduces Learns how to continually learn (OML) i.e. learns how to do online updates without forgetting. 
Learning MAML in a Meta continual learning way slows down forgetting

Lifelong Reinforcement Learning

Formulates an online learning procedure that uses SGD to update model parameters, and an EM with a Chinese restaurant process prior to develop and maintain a mixture of models to handle non-stationary task distribution

Continual Generative Modeling

Introduces unsupervised continual learning (no task label and no task boundaries) 
Extensive evaluation of CL methods for generative modeling

Applications

a healthcare-specific replay-based method to mitigate destructive interference during continual learning 
method for compositional continual learning of sequence-to-sequence models 
HTM applied to real-world anomaly detection problem 
HTM applied to a prediction problem of taxi passenger demand

Thesis

Workshops

About

Relevant papers in Continual Learning

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • TeX92.9%
  • Python7.1%