Science Limelight

  Binary Context: A Practical Way to Think About Neural Networks Much of deep learning can be understood in surprisingly simple terms. Rather than thinking about neurons "computing features," it is often more useful to think of them as selecting which computation should happen next . The selected computation is almost always linear. The interesting part is not the linear mapping itself. The interesting part is how the network chooses which linear mapping to use. Binary context Consider a layer with a binary context vector. The context might come from: ReLU activations (positive or zero) threshold units locality sensitive hash (LSH) bits routing networks mixture-of-experts gates any other binary routing mechanism The binary vector does not directly solve the problem. Instead it answers a much simpler question: Which parameters should participate in the next computation? Once those parameters are selected, the layer performs an ordinary linear mapping. So every layer is essenti...

 

  Atlas LSH Neural Networks: Geometry as Context Most neural networks perform essentially the same computation for every input. Every layer applies the same collection of parameters, regardless of what the input actually is. Atlas LSH neural networks explore a different idea: use the geometry of the input to determine which computation should be performed. The first step is to take a compact geometric sample of the input using locality-sensitive hashing (LSH). Each LSH bit can be viewed as asking a simple geometric question about the input vector—for example, which side of a randomly oriented hyperplane it lies on. A few hundred such bits form a sparse fingerprint describing the input's approximate location in a high-dimensional space. The important point is that these bits are not trying to represent the input in detail. Instead, they capture context . Similar inputs tend to produce similar bit patterns, so the LSH serves as a geometry sampler that identifies the neighborhood in ...

  This interactive sketch explores how a binary diagonal matrix D interacts with the Walsh–Hadamard mixing matrix H.  Each entry of D can be flipped between +1 and -1, dividing the nodes into two groups.  The code visualizes two derived operators:  ●​ X = HD + DH  ●​ Y = HD - DH  These have a surprisingly clean interpretation:  ●​ X keeps interactions between nodes with the same sign (“agreement”).  ●​ Y keeps interactions between nodes with opposite signs (“disagreement”). Because the Walsh–Hadamard matrix contains only ±1 entries, the resulting matrix values become simple:  ●​ reinforcement: +2  ●​ cancellation: 0  ●​ opposite reinforcement: -2  The graph views make this intuitive: ●​ The graph of X splits into same-sign clusters.  ●​ The graph of Y becomes a bipartite interaction graph between opposite-sign groups.  As entries of D are flipped, the system transitions between:  ●​ uniform structure (all signs e...

Brief Introduction to Extreme Learning Machines (ELM) Extreme Learning Machines (ELMs) are a type of feedforward neural network known for fast training speed and good generalization performance. The contain 2 active layers. A fixed random layer with activation functions and an entirely linear read-out layer.  Here's a point-wise introduction: Type: Single-hidden layer feedforward neural networks (SLFNs). Key Idea: Input weights and biases in the hidden layer are randomly assigned and not updated during training. Training: The read-out layer weights are analytically determined using least squares (no iterative backpropagation).  Single shot least square. Advantage: Extremely fast training compared to traditional neural networks. Application Areas: Regression, classification, clustering, and feature learning tasks. Limitation: Performance depends on hidden layer size and random initialization. Current State of the Art (as of 2025) ELM Variants: ...

 Java Machine Learning Collection: An archive of machine learning code and general algorithms written in Java. Neural Networks Associative Memory Random Number Generators Random Projections Image Compression General Algorithms The Java source code is written in the style of the Numerical Recipies books.  Short and simple, for the most part. Java Collection >>>>> Download