CCSLM as Hierarchical Associative Memory

 

When we talk about CCSLM, we tend to talk about linear mappings.

But there is another way to look at those same mappings that may have been hiding in plain sight: a linear mapping is also an associative memory.

A matrix does not merely transform numbers. It stores associations between patterns. And those memories have engineering properties: capacity limits, noise sensitivity, interference, redundancy, and error-correction behavior.

These considerations were once central to the study of associative memory, but they can become strangely distant from view when we look at modern neural networks primarily as layers of learned nonlinear functions.

CCSLM brings them back into the foreground.

A CCSLM network is a sequence of context-selected linear mappings:

Each mapping can therefore be considered an associative memory operating on the representation produced by the preceding mappings.

This immediately raises an important question: how observable is the original input as we move through the hierarchy?

If successive transformations discard information about the original input, then later associative memories have progressively less information available from which to make their associations. Observability therefore becomes an important engineering consideration. We should not only ask what a later layer has learned, but also what information from the original input remains available to it.

We should also be cautious about assuming exactly what is stored at each level. Early memories may respond to relatively local structures and later memories to more abstract combinations, but CCSLM gives us an opportunity to measure what is actually being stored rather than simply assuming it.

And this opens up a larger research program.

What is the capacity of each linear associative memory? How does noise affect recall? How much redundancy is available? What happens when the memory is operated below capacity? Can that spare capacity provide error correction? How does interference accumulate through a hierarchy? How does context alter these properties?

These are not necessarily new questions. They are questions that associative-memory research has wrestled with for decades.

Perhaps what is new is applying them systematically to modern context-selected networks.

CCSLM therefore gives us a useful change of viewpoint:

A deep network can be regarded as a hierarchy of associative memories, with context selecting which memories perform the successive transformations.

Once we see the matrices as memories rather than merely weights, a whole collection of engineering questions comes back into view.

And some of those questions may have been sitting there in the mathematics all along.