Dimensionality reduction using neural networks. Neural networks can be shown to perform a very similar sort of dimensionality reduction, if they are forced to “auto-associate” an input vector with an equivalent output vector, compressing the information through a hidden layer of k units [DeMers and Cottrell, 1993]. One important difference, however, is that neural networks spread the variance evenly across all hidden units, rather than concentrating most of it along the first eigenvector, etc.