Atlas LSH Neural Networks: Geometry as Context
Atlas LSH Neural Networks: Geometry as Context Most neural networks perform essentially the same computation for every input. Every layer applies the same collection of parameters, regardless of what the input actually is. Atlas LSH neural networks explore a different idea: use the geometry of the input to determine which computation should be performed. The first step is to take a compact geometric sample of the input using locality-sensitive hashing (LSH). Each LSH bit can be viewed as asking a simple geometric question about the input vector—for example, which side of a randomly oriented hyperplane it lies on. A few hundred such bits form a sparse fingerprint describing the input's approximate location in a high-dimensional space. The important point is that these bits are not trying to represent the input in detail. Instead, they capture context . Similar inputs tend to produce similar bit patterns, so the LSH serves as a geometry sampler that identifies the neighborhood in ...