Sparse Predictive Hierarches: An Alternative to Deep Learning

Abstract

This tutorial serves to describe an alternative paradigm to the highly popular Deep Learning, by describing methods which avoid backpropagation, i.i.d sampling, batches, and dense representations in favor of biologically-inspired, sparse, online/incremental learning with entirely local operations. It seeks to show that such methods can work in practice by using it on compute-constrained and robotics tasks. We hope to inspire others to seek other alternatives to Deep Learning, and discuss how alternative paradigms can overcome many of the limitations of Deep Learning such as catastrophic interference and high compute costs.