Science Limelight

Context-Controlled Linear Mappings Using Binary Sign Flips and Fast Walsh–Hadamard Transforms 1. Generating a constrained binary context The first component constructs a binary context vector from analog control signals and fixed Gaussian random vectors. Begin with analog signals, for example random projections of the input. Associate each signal with a fixed random vector whose components are sampled from a Gaussian distribution. Multiply each random vector by its corresponding analog signal and sum the weighted vectors. Binarize the resulting vector into +1 and -1 values. The result is a binary context that controls the signs of selected input dimensions. This construction provides a way to restrict the binary context space. Although an m-dimensional binary vector has 2ⁿ possible patterns in principle, a low-dimensional analog control process will reach only a subset of them. The context mechanism can therefore introduce useful variation without necessarily exposing the trainable map...

 

Bidirectional initial training of (Atlas LSH or similar neural networks) for nice preconditioning. https://archive.org/details/bidirectional-conditioning-of-context-selected-linear-networks

 

  https://archive.org/details/ccslm-matrix-products-alignment-rank-and-energy