Biological Systems and Continual Learning Methods
==Biological systems inspire continual learning by combining multiple mechanisms—spanning synaptic plasticity and consolidation, modular functional organization, and explicit memory systems—to learn from sequential, c...
==Biological systems inspire continual learning by combining multiple mechanisms—spanning synaptic plasticity and consolidation, modular functional organization, and explicit memory systems—to learn from sequential, correlated experience while limiting interference (catastrophic forgetting).==[:cite[1]{ln=1}], [:cite[1]{ln=2}], [:cite[1]{ln=3}], [:cite[2]{ln=1}], [:cite[3]{ln=3}], [:cite[3]{ln=5}], [:cite[3]{ln=6}] 1) Complementary learning systems (CLS): fast episodic memory + slow statistical learning Brains are described as using two complementary systems : a neocortical system that gradually acquires structured knowledge and a hippocampal system that rapidly stores specifics of individual experiences.[:cite[4]{ln=2}], [:cite[5]{ln=1}], [:cite[5]{ln=2}] This dual system idea motivates AI designs that separate fast, instance based storage from slow, parametric generalization , rather than forcing one set of weights to serve both roles.[:cite[4]{ln=2}], [:cite[6]{ln=5}], [:cite[6]{ln=6}] 2) Replay + interleaving: rehearsal to prevent interference CLS theory highlights hippocampal replay as a mechanism that can reactivate new experiences back to neocortex, interleaved with other experiences, enabling integration while reducing disruption to existing knowledge.[:cite[7]{ln=2}], [:cite[7]{ln=3}], [:cite[7]{ln=4}] This maps directly onto modern continual learning and RL practice: experience replay stores past transitions and trains on random subsets to “break up” harmful correlations and stabilize learning.[:cite[8]{ln=1}], [:cite[8]{ln=2}], [:cite[8]{ln=3}], [:cite[9]{ln=1}], [:cite[9]{ln=2}] Neuroscience inspired AI reviews explicitly frame the DQN replay buffer as a primitive hippocampus enabling “complementary learning” in silico.[:cite[11]{ln=1}], [:cite[10]{ln=2}], [:cite[10]{ln=3}] 2a) Selective / prioritized replay (salience weighted memory) Biology suggests replay is not uniformly sampled: replay can be modulated by reward or novelty to rebalance experience toward agent goals.[:cite[12]{ln=3}], [:cite[5]{ln=3}] Consistent with this, work connecting replay to continual learning notes gains from biasing replay toward significant events (e.g., high reward prediction error), echoing ideas about hippocampal reweighting of experiences.[:cite[13]{ln=3}], [:cite[13]{ln=4}], [:cite[14]{ln=3}], [:cite[14]{ln=4}] 3) Synaptic consolidation → regularization methods (protect important weights) At the synaptic level, biology exhibits mechanisms that protect old knowledge from interference while new learning proceeds.[:cite[16]{ln=2}], [:cite[15]{ln=2}] This inspires regularization based continual learning , where changes to parameters important for prior tasks are penalized to reduce forgetting.[:cite[17]{ln=1}], [:cite[17]{ln=2}], [:cite[17]{ln=3}] A prominent example is elastic weight consolidation (EWC) , which is explicitly described as neuroscience inspired: it slows learning on a subset of important weights, “anchoring” them to previously found solutions so multiple tasks can be learned without increasing network capacity.[:cite[18]{ln=1}], [:cite[18]{ln=2}], [:cite[18]{ln=3}], [:cite[19]{ln=5}], [:cite[19]{ln=7}] 4) Modularity + sparsity: isolating learning updates to reduce interference Both neuroscience and continual learning reviews emphasize that biological brains are modular , with distinct but interacting subsystems (e.g., memory vs. motor control), and that modularity supports specialization.[:cite[20]{ln=3}], [:cite[20]{ln=4}] This motivates modular continual learning architectures that set boundaries between task specific parameters (with optional shared parameters for transfer), reducing interference vs. monolithic networks.[:cite[21]{ln=4}], [:cite[22]{ln=1}], [:cite[22]{ln=2}] Relatedly, sparsity (sparse activations/gradients) is highlighted as a way to reduce forgetting because fewer task sensitive parameters are changed per update, and it can even yield emergent modules without predefining a modular structure.[:cite[20]{ln=4}], [:cite[23]{ln=1}], [:cite[23]{ln=2}], [:cite[23]{ln=3}] 5) Explicit memory systems: write/read memory to support long term retention Continual learning work grounded in biology treats memory as central: memory systems help protect learning and adaptation over long lifetimes.[:cite[24]{ln=2}] Accordingly, memory based methods in AI “write experience to memory” to avoid forgetting.[:cite[21]{ln=5}] Replay/rehearsal approaches—in which a buffer of past observations is sampled alongside current data—are described as simple and effective at reducing forgetting, although scaling is a challenge.[:cite[25]{ln=1}], [:cite[25]{ln=3}] Biologically motivated critiques also note that perfect storage of all pristine observations is unrealistic, motivating compressed representations (anchors/exemplars) or generative memory that generates rehearsal data rather than storing samples...