Concatenated Context Selected Linear Mappings

 

Concatenated Context Selected Linear Mappings

Many computational systems can be described as repeatedly applying mathematical transformations to data. One particularly simple and surprisingly expressive architecture can be built from just two ingredients:

  • A method for selecting a context.

  • A collection of linear mappings associated with those contexts.

The computation then becomes a sequence of Concatenated Context Selected Linear Mappings (CCSLMs).

This description intentionally says nothing about neural networks, activation functions, or even learning. It simply describes a family of computational systems.

The Basic Idea

Imagine you have many linear transformations available.

Instead of always using the same transformation, a context function examines the current state and chooses which one should be applied.

Mathematically,


y = Ac(x)x

where

  • x is the current state,

  • c(x) selects a context,

  • Ac(x) is the linear mapping associated with that context.

The interesting part comes from repeating this process.

Concatenation

A complete computation is simply many of these stages connected together.


xk+1 = Ack(x or xk)xk

Each stage

  1. observes the current state,

  2. selects a context based on xk or x (the overall input to the system) or some other concept such as time of day.

  3. applies the corresponding linear mapping,

  4. passes the result onward.

The overall computation is therefore a concatenation of context-selected linear operators.

Why Context?

If every stage always used the same matrix, the entire system would collapse into one large linear mapping.

Context selection changes everything.

Different inputs may travel through different computational routes, allowing the overall system to exhibit highly nonlinear behaviour while every individual mapping remains perfectly linear.

One may think of context as choosing which page of an enormous mathematical atlas should be used for the next step of computation.

Local Simplicity

Each individual mapping can remain remarkably simple.

Within a particular context, the desired computation may be well approximated by a linear transformation.

Rather than asking one enormous mapping to solve every possible case, many specialized mappings each solve a smaller, easier problem.

The complexity moves from the mappings themselves into the mechanism that selects them.

Factorizing Complexity

This viewpoint naturally factorizes computation into two distinct problems.

First:

"What situation am I currently in?"

Second:

"Given that situation, what linear mapping should I use?"

Separating these responsibilities often leads to a much cleaner conceptual model than attempting to perform everything with one universal operator.

Context Can Be Defined Many Ways

The context function is deliberately left unspecified.

Examples include

  • threshold comparisons,

  • binary signatures,

  • locality-sensitive hashes,

  • geometric partitions,

  • symbolic rules,

  • decision trees,

  • finite-state machines,

  • lookup tables,

  • externally supplied control signals.

The CCSLM framework does not depend upon any particular method.

Multiple Layers

Stacking many context-selected mappings creates progressively richer computations.

Earlier stages may perform broad transformations.

Later stages may refine those results using increasingly specialized contexts.

Although every individual operation remains linear, the sequence as a whole becomes capable of expressing extremely complicated functions.

Training

The architecture itself does not require any particular method for determining either the contexts or the linear mappings.

They may be designed manually, optimized numerically, generated procedurally, evolved, or learned from data.

One widely used optimization method is backpropagation, which can adjust the parameters of both the context-selection mechanisms (where differentiable) and the linear mappings so that the overall computation better matches desired outputs.

Other optimization methods are equally compatible with the framework.

A Different Way to Think About Computation

CCSLMs encourage a different perspective.

Instead of viewing computation as a mysterious nonlinear process, they suggest viewing it as a journey through a sequence of locally linear coordinate systems.

Each context identifies where you are.

Each linear mapping describes how to move within that region.

The overall computation is therefore not one enormous transformation but a carefully organized chain of context-dependent linear approximations.

Conclusion

Concatenated Context Selected Linear Mappings provide a simple abstraction for a surprisingly broad class of computational systems.

They consist only of repeated applications of:

  • context selection,

  • linear transformation,

  • concatenation.

From these modest ingredients, highly sophisticated computations can emerge.

Whether the contexts are determined by geometric partitions, hash functions, logical rules, or learned decision mechanisms, the underlying principle remains unchanged:

Computation is performed by repeatedly selecting an appropriate local linear mapping based on context and concatenating those mappings into a coherent whole.


https://archive.org/details/@seanc4s

Comments

Post a Comment

Popular posts from this blog

Neon Bulb Oscillators

Image
Neon Tube Oscillators  Fig. 1 High voltage to headphones, don't do this in modern times. Very high impedance headphones are implied. A neon bulb is useful in many ways in an amateur station, but there are undoubtedly many amateurs who are not aware of the fact that the device can be made to oscillate at either audio or radio frequencies. Fig. 1 shows how a single bulb may be hooked up as an audio frequency oscillator. The L-C circuit is formed by the headphones and a condenser, and by varying the capacity the circuit can be made to oscillate at frequencies from a few cycles per second up to the highest audio frequencies.  The variable resistor in series with the bulb and battery controls the strength of oscillation, and also controls the frequency to some extent. A 1 nf variable condenser will be sufficient to cover most of the audio range.  Fig. 2 The circuit in Fig. 2 may be used when an RF oscillator is to be modulated at audio frequencies. The transformer may be an or...
Read more

A practical way to view neural networks (whiteboard view)

Image
 A whiteboard view: You can take a decision matrix view of ReLU neural networks. Instead of ReLU(Wx) you create a diagonal matrix with binary 1 or 0 entries according to the ReLU decisions (x>=0?) Then conceptually a layer is DWx. Where D is doing row selection on W. And in fact D does column selection on the weight matrix in the next layer. That is very coarse parameter selection which is linear mapping selection.
Read more

23 Circuits you can Build in an Hour - Free Book

Image
Unleash your inner electronics wizard with 23 captivating circuits you can build in a jiffy! Tired of staring at daunting circuit diagrams that seem to go on forever? Craving hands-on experience that will ignite your passion for electronics? Look no further than this electrifying collection of 23 circuits you can build in an hour or less! From the delightfully simple to the mind-bendingly clever, these circuits are guaranteed to spark your curiosity and inspire further experimentation. Whether you're a seasoned pro or a budding hobbyist, you'll find something to tantalize your taste buds in this smorgasbord of electronic delights. Get the book now, it's free. The Lost SuperFoods (sponsored) Crash GPT course (sponsored) From Zero To Off-Grid Hero (sponsored)
Read more