On the Relationship Between Input Sparsity and Noise Robustness in Hierarchical Temporal Memory Spatial Pooler

Abstract

Hierarchical Temporal Memory - Spatial Pooler is a cortical learning algorithm inspired by the biological functioning of the neocortex. It is responsible for the sparse encoding of spatial patterns used as an input for further processing inside of a Hierarchical Temporal Memory (HTM). During the learning process, the Spatial Pooler groups spatially similar inputs into the same sparse distributed representation (SDR) memorized as a set of active mini-columns. The role of SDR generated by the learning process of the Spatial Pooler is to provide an input for learning of sequences inside of the HTM. One of the features of the Spatial Pooler is also the robustness to noise in the input. This paper summarizes the work in progress, which analyses the relationship between the encoding of the input pattern and the robustness of the memorized pattern against noise. In this work many synthetic input patterns with different sparsity were used to set the hypothesis, which claims that SP robustness against the noise in the input depends on the sparsity of the input. To validate the hypothesis, many random input vectors with a large portion of noise were generated. Then the change of SDR output was compared with the change of input by the given portion of noise. It was shown that the SDR output change is very small in comparison to change of the input by adding of a large portion of the noise in the input. By adding of a significant portion of the noise to the input, the learned output remains almost unchanged. This indicates a great robustness to noise. Experiments show that the robustness against noise of the Spatial Pooler directly depends on the sparsity of the input pattern. Preliminary tests suggest implementation of a boosting mechanism of the input to improve the robustness against noise.